Build your own Forth for Microchip PIC (Episode 837)
none Byron Jeff
I wanted to introduce myself and throw out some ideas in what I perceive
to be as a gap in the Forth continuum. Any feedback would be
appreciated. Sorry for the length but I've been doing a lot of thinking.
Feel free to snip whatever you like. I have a lot of setup here, you can
skip down about 150 lines if you are interested in only my questions.
I'm an IT professor at Clayton State University in the Atlanta area. So
my interest is primarily academic. I'm also a tool builder. The sense of
understanding and control I get when working with my own tools is
appealing. I'm also an Open Source and Linux guy. So I'll happily use
others' tools so long as I have the ability to adapt them to my needs.
Finally I'm a embedded systems guy that has been using Microchip Pics
for development of small scale embedded systems projects for years. I'm
well aware that the architecture is quirky and limited, but the
confluence of price, documentation, and simplicity has made it a choice
of mine for awhile. Plus Microchip offers free samples.
So where does Forth fit into this? Truthfully I wish that I had come
across it 15 or 20 years ago. The simplicity and elegance of Forth's
language/system concepts (stack based, generating/incremental compilation,
and interactivity) are intriguing. Forth came across my screen a couple
of weeks ago while I was taking a look at the Parallax Propeller:
http://www.parallax.com/propeller/index.asp
A $12 8 CPU multiprocessor embedded system that has a collective 160
MIPS of processing power and comes in a 40 pin DIP suitable for
prototyping is a wonder to behold. But the culture around the part also
illustrates why I like building my own tools. Parallax developed a high
level object language called Spin around the part. They embedded an
optimized bytecode interpreter for the language into the part. The
disheartening thing is that they then build the compiler and IDE
environment for Spin specifically for Windows. Worse is that the
source and specifications for the environment are closed. In short to
program the part in Spin you must use their tool on their platform.
Sigh.
Fortunately another developer, Cliff Biffle, who was in the same boat (a
Mac OSX guy) solved the problem by hosting Forth on the chip.
PropellerForth:
http://www.cliff.biffle.org/software/propeller/forth/index.php
requires only a terminal interface to get development done. In addition
Cliff has added words to facilitate multiprocessor tasking. So it's
possible to run multiple simulteaneous tasks on each of the 8 Propeller
cogs (CPUs).
Cliff's contribution got the ball rolling for me. Searching around let
me pretty quickly to Starting Forth, Thinking Forth, and Steven Pelc's
Programming Forth. I also installed gforth on my Linux box and started
noodling.
Fortunately for me stack based programming is old hat. Over the years
I've built interpreted languages for processing web pages, teaching
beginning programming, and programming PICs (combination of HLL compiler
along with a bytecode interpreter). In each I utilized the concept of an
execution data stack for processing expressions and subroutine calls and
parameters (when I implemented subroutines). But each time I approached
the high level language in the traditional compilation, parsing, code
generation/interpretation view of most high level languages. Forth's
view of the world instantly clicked with me.
So the next question for me was "What about forth for PICs?". As a long
time contributor to the Piclist mailing list, I can never remember seeing
a Forth discussion. Google of course popped out a handful of what are
the usual suspects: Picforth: http://www.rfc1149.net/devel/picforth and
Mary: http://mary.pepix.net. I'm limiting my discussion to Forths for
the 16F PIC family as that's my primary focus.
I played with Picforth for a few and quickly realized that it was in
fact a Forth compiler. I was disappointed because I feel that takes away
one of Forth strengths: interactivity and incremental compilation. For
years I've used Pic assembly and other languages such as Jal, and my own
PIC HLL called NPCI for development. As compilers you fall into the
traditional edit -> compile -> download -> test cycle. That's exactly
the cycle I wish to get away from. The ideal Forth for embedded
development would be interactive, with incremental development, and self
hosted so that when the application is finished, it's all onboard the
target. So the game's afoot!
c.l.f is a rich resource of discussion on the subject. Searches for PIC
Forth, minimal forth, target forth, and embedded forth came up with mega
threads of discussions that I've been pouring over. A couple of
observations:
1. A specification for hosting Forth with a minimal kernel would be
instructive for tool developers because it would abstract what one needs
to implement Forth in a particular environment. One of my maxims to my
students is "Make it work. Then make it pretty." In Forth's case I'd
change that to "Make it work. Then make it fast." The concept of minimal
kernels always gets shot down because of the resulting speed of the
result. However, if a developer can get Forth working on the target,
then optimizing words to make it faster can be done incrementally as you
move forward. One crack at this was done in this thread:
with the specification here:
http://www.quirkle.com/misc/forth.htm
While I think Jim has the wrong target audience, the general Forth
enthusiast, the idea has merit if targeted towards tool builders.
2. Such a specification should be architecture neutral. Specs for
eforth, hforth, and retroforth (which are often suggested when the
question "How do I build my own Forth?" come up) that I've seen seem to
be a bit too tightly coupled to the X86 platform. A forth specification
that factors out the primitives would be helpful. A tool builder is
probably going to target something that has not in fact been done yet.
3. For the purposes of embedded development, that an environment that
appears as close to self hosting as possible would be desirable.
Frank Sergeant idea is really close to being the jackpot. What could be
a simpler kernel than 3 instruction Forth?
http://pygmy.utoh.org/3ins4th.html
It's almost like Frank was thinking about me coming along 16 years later
when he wrote this. Elizabeth gives an explanation of the tethered Forth
approach to embedded systems development here:
In evaluating Frank's discussion I perfectly agree with why the
microkernel approach is the right one, put the simplest, easiest to test
executive on the target will get you going the fastest. Couple that with
a full fledged forth environment on the host, you can get rolling pretty
quickly.
However, I disagree with Frank's assertion that you actually have Forth
on the target. There's a gap between Frank's bytecode to execute
arbitrary code on the target and actually running Forth on the target.
So at long last we get to the gap I referred to in my opening. Frank's 3
instruction executive facilitates getting an embedded system connected
to a Forth host so you can interactively develop forth on the host and
exercise the target. But his XCALL instruction facilitates calling
arbitrary code on the target, not Forth code.
So the question is "Given an embedded micro, what is the minimum forth
environment required to execute Forth words?". If we can answer this
question then along with the memory read/write instructions it should be
possible to migrate Forth words (both code and compiled) to the target
and have them executed there. If you can incrementally migrate the
application to the target while developing it, then when you finish you
can simply disconnect the target from the host and run the entire
application on the target. All the while during development, you have
the full facilities of the hosted forth environment available for you to
interactively develop the application.
Brad Rodriguez's Moving Forth articles gives a pretty good idea of how
to pull it off:
http://www.zetetics.com/bj/papers/moving1.htm
Brad is of course very interested in how to implement such a kernel in
an optimized way. But again "make it work, then make it fast." comes to
mind. In my estimation one needs two sets of items to execute forth words:
1. The forth virtual machine including the standard registers and
stacks.
2. Three critical words: ENTER, EXIT, and NEXT.
In short create enough Forth to execute the inner interpreter and you
can get going.
It seems to me that neither of those two items are so complicated that
they could not be easily put together for a target. And Frank's 3 word
forth implements a inner-inner interpreter that can be used to create
the inner interpreter required to run forth words on the target.
Only one piece of the puzzle is missing at this point. Elizabeth
discusses in her post above that the XTL transfers the stack between the
host and the target. In short it implements a form of distributed
execution where you muster the stacks for RPC. In doing so one can run a
application with a set of words distributed between the host and the
target.
So I envision an environment where Forth words are quick interactively
developed on the host. Once satisfied they work, they are incrementally
compiled on the host and transferred to the target. applications can be
a set of shared words between the two until complete at which point all
needed words are transferred to the target, completing the app.
There are still details that need to be worked out such as how to
differentiate between local words and remote words in both systems and
how to facilitate transferring the stacks between the two. In both cases
solutions should be geared towards simplifying the target.
I do believe that the minimal kernel still needs to be specified. It'll
give a list of words that needs to be implemented, tested, and
incorporated in the initial kernel. I can help here because as I stated
earlier my NPCI bytecode interpreter implements a stack based virtual machine
and is written in pic assembly for the 16F family. It has debugged code
for all 16 bit arithmetic and logical ops, number processing for stacks,
and conditional/unconditional jumps. It already implements an IP for the
bytecode.
I look forward to hearing your comments on these thoughts.
TTYL,
BAJ
Elizabeth D Rather
> Hi folks,
>
> I wanted to introduce myself and throw out some ideas in what I perceive
> to be as a gap in the Forth continuum. Any feedback would be
> appreciated. Sorry for the length but I've been doing a lot of thinking.
> Feel free to snip whatever you like. I have a lot of setup here, you can
> skip down about 150 lines if you are interested in only my questions.
>
> c.l.f is a rich resource of discussion on the subject. Searches for PIC
> Forth, minimal forth, target forth, and embedded forth came up with mega
> threads of discussions that I've been pouring over. A couple of
> observations:
>
> 1. A specification for hosting Forth with a minimal kernel would be
> instructive for tool developers because it would abstract what one needs
> to implement Forth in a particular environment. One of my maxims to my
> students is "Make it work. Then make it pretty." In Forth's case I'd
> change that to "Make it work. Then make it fast." The concept of minimal
> kernels always gets shot down because of the resulting speed of the
> result. However, if a developer can get Forth working on the target,
> then optimizing words to make it faster can be done incrementally as you
> move forward. One crack at this was done in this thread:
>
> http://tinyurl.com/22c7sn
>
> with the specification here:
>
> http://www.quirkle.com/misc/forth.htm
>
> While I think Jim has the wrong target audience, the general Forth
> enthusiast, the idea has merit if targeted towards tool builders.
>
> 2. Such a specification should be architecture neutral. Specs for
> eforth, hforth, and retroforth (which are often suggested when the
> question "How do I build my own Forth?" come up) that I've seen seem to
> be a bit too tightly coupled to the X86 platform. A forth specification
> that factors out the primitives would be helpful. A tool builder is
> probably going to target something that has not in fact been done yet.
There is at least a draft proposal for a cross-compiler addendum to ANS
Forth, at ftp://ftp.forth.com/pub/ANSForth. The actual proposed
standard is in XCtext5.doc, and non-normative explanatory appendices in
XCapp5.doc. There are also pdf forms. The '5' indicates that this is
the 5th draft, following an extended public review period.
(NOTE: at this moment, 9:30 am Hawaii time on 6/23, that link isn't
working. I'm trying to find out why and get it fixed. Meanwhile, if
you really want the docs now, email me erather {at} forth {dot} com)
> 3. For the purposes of embedded development, that an environment that
> appears as close to self hosting as possible would be desirable.
>
> ... Elizabeth gives an explanation of the tethered Forth
> approach to embedded systems development here:
>
> http://tinyurl.com/yv6z99
Yes, this abbreviated description is consistent with the draft standard
above. It was developed jointly by FORTH, Inc. and MPE in the late
90's, and both FORTH, Inc. and MPE have been developing and using
cross-compilers that work this way since then. It's very mature and
well-understood technology.
That said, improvements do occur. By hosting the actual compilation on
a powerful desktop, you can do things that are both difficult and
inappropriate on a limited target, such as compiling optimized target
machine code (which we do). The compile-to-optimized-machine-code
approach not only generates much faster code than the traditional
indirect-threaded approach, it's comparable in size on even small
targets such as the 8051, and significantly smaller on larger ones such
as the 68K family.
...
> So the question is "Given an embedded micro, what is the minimum forth
> environment required to execute Forth words?". If we can answer this
> question then along with the memory read/write instructions it should be
> possible to migrate Forth words (both code and compiled) to the target
> and have them executed there. If you can incrementally migrate the
> application to the target while developing it, then when you finish you
> can simply disconnect the target from the host and run the entire
> application on the target. All the while during development, you have
> the full facilities of the hosted forth environment available for you to
> interactively develop the application.
Well, obviously that depends both on the overall architecture you're
using (e.g. compiling optimized machine code vs. supporting an indirect
threaded model), not to mention the needs of the applications you're
going to be using it for. For example, if you know you're going to be
doing a lot of certain kinds of functions (e.g. string management or
double-length arithmetic) you'd go with code implementations of those
key functions from the get-go, rather than having to go back and
optimize them later.
> ...In my estimation one needs two sets of items to execute forth words:
>
> 1. The forth virtual machine including the standard registers and
> stacks.
>
> 2. Three critical words: ENTER, EXIT, and NEXT.
>
> In short create enough Forth to execute the inner interpreter and you
> can get going.
Except that if you compile to code NEXT goes away, and ENTER/EXIT are
merely subroutine call/return machine instructions.
> It seems to me that neither of those two items are so complicated that
> they could not be easily put together for a target. And Frank's 3 word
> forth implements a inner-inner interpreter that can be used to create
> the inner interpreter required to run forth words on the target.
Indisputably the traditional indirect-threaded model is easier for
newbies to implement, but may not be satisfying for challenging or
time-critical applications.
> Only one piece of the puzzle is missing at this point. Elizabeth
> discusses in her post above that the XTL transfers the stack between the
> host and the target. In short it implements a form of distributed
> execution where you muster the stacks for RPC. In doing so one can run a
> application with a set of words distributed between the host and the
> target.
>
> So I envision an environment where Forth words are quick interactively
> developed on the host. Once satisfied they work, they are incrementally
> compiled on the host and transferred to the target. applications can be
> a set of shared words between the two until complete at which point all
> needed words are transferred to the target, completing the app.
That tends not to be satisfactory, because most embedded apps feature
custom I/O, so real target-specific words can't be tested on the host
without your having to develop simulators for the I/O functions. It's a
lot easier to just go with a fully functional XTL and do all your
testing on the target. Among other benefits, that means you can use
Forth to debug your target hardware, which is wonderful.
> There are still details that need to be worked out such as how to
> differentiate between local words and remote words in both systems and
> how to facilitate transferring the stacks between the two. In both cases
> solutions should be geared towards simplifying the target.
That differentiation is typically done with wordsets (formerly known as
vocabularies). The draft standard identifies "scopes" of words for the
host, cross-compiler, and target; they are usually implemented with
wordsets, although the draft standard doesn't mandate any particular
implementation strategy.
> I do believe that the minimal kernel still needs to be specified. It'll
> give a list of words that needs to be implemented, tested, and
> incorporated in the initial kernel. I can help here because as I stated
> earlier my NPCI bytecode interpreter implements a stack based virtual machine
> and is written in pic assembly for the 16F family. It has debugged code
> for all 16 bit arithmetic and logical ops, number processing for stacks,
> and conditional/unconditional jumps. It already implements an IP for the
> bytecode.
Well, that sounds useful. Frankly, the reason so little Forth work has
been done on the PIC family (and PICForth is so minimal) is that its
instruction set is really a bit below the mark for a reasonable
implementation. The PIC18 should be do-able, although we haven't yet
seen much demand for it.
> I look forward to hearing your comments on these thoughts.
I think it would be a good investment of your time to take a hard look
at existing mature Forth cross-compilers. You can get a CD with
extensive docs and links to free evaluation versions of our SwiftX
cross-compilers for many chips (8051, 68HCS08, 68HC11, MSP430, AVR, ARM,
68HC12, 68K family, Coldfire, more) for only $15. You can get supported
boards for most of these processors very inexpensively. The evaluation
compilers are limited only in the size of the target app you can
develop, so you can exercise them and learn a lot. For more info go to
http://www.forth.com/embedded/index.html.
Cheers,
Elizabeth
--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310-491-3356
5155 W. Rosecrans Ave. #1018 Fax: +1 310-978-9454
Hawthorne, CA 90250
http://www.forth.com
"Forth-based products and Services for real-time
applications since 1973."
==================================================
none Byron Jeff
Elizabeth D Rather <erath...@forth.com> wrote:
>none Byron Jeff wrote:
[Snippage]
>There is at least a draft proposal for a cross-compiler addendum to ANS
>Forth, at ftp://ftp.forth.com/pub/ANSForth. The actual proposed
>standard is in XCtext5.doc, and non-normative explanatory appendices in
>XCapp5.doc. There are also pdf forms. The '5' indicates that this is
>the 5th draft, following an extended public review period.
>(NOTE: at this moment, 9:30 am Hawaii time on 6/23, that link isn't
>working. I'm trying to find out why and get it fixed. Meanwhile, if
>you really want the docs now, email me erather {at} forth {dot} com)
I'll take a look. I can wait.
>> 3. For the purposes of embedded development, that an environment that
>> appears as close to self hosting as possible would be desirable.
>> ... Elizabeth gives an explanation of the tethered Forth
>> approach to embedded systems development here:
>Yes, this abbreviated description is consistent with the draft standard
>above. It was developed jointly by FORTH, Inc. and MPE in the late
>90's, and both FORTH, Inc. and MPE have been developing and using
>cross-compilers that work this way since then. It's very mature and
>well-understood technology.
Then that's a draft I definitely want to take a look see.
>That said, improvements do occur. By hosting the actual compilation on
>a powerful desktop, you can do things that are both difficult and
>inappropriate on a limited target, such as compiling optimized target
>machine code (which we do). The compile-to-optimized-machine-code
>approach not only generates much faster code than the traditional
>indirect-threaded approach, it's comparable in size on even small
>targets such as the 8051, and significantly smaller on larger ones such
>as the 68K family.
I'm not necessarily opposed to the optimized-machine-code approach. It's
just that with the PIC based forths I've seen so far, there's no
incremental way to do it. So it reverts back to the traditional compile
the whole app, download the whole app, test cycle that associated with
traditional HLLs. And the target in this instance cannot be programmed
at wire speed. So downloading a entire program to test takes a while.
It's not like a PC where compiling is virtually instantaneous.
>> So the question is "Given an embedded micro, what is the minimum forth
>> environment required to execute Forth words?". If we can answer this
>> question then along with the memory read/write instructions it should be
>> possible to migrate Forth words (both code and compiled) to the target
>> and have them executed there. If you can incrementally migrate the
>> application to the target while developing it, then when you finish you
>> can simply disconnect the target from the host and run the entire
>> application on the target. All the while during development, you have
>> the full facilities of the hosted forth environment available for you to
>> interactively develop the application.
>Well, obviously that depends both on the overall architecture you're
>using (e.g. compiling optimized machine code vs. supporting an indirect
>threaded model), not to mention the needs of the applications you're
>going to be using it for. For example, if you know you're going to be
>doing a lot of certain kinds of functions (e.g. string management or
>double-length arithmetic) you'd go with code implementations of those
>key functions from the get-go, rather than having to go back and
>optimize them later.
Obviously there are design issues there. The problem is that if you're
starting with a blank slate per-se, then having the ability to have a
blended approach (a combo of optimized native and high level compiled
code) to development is a win.
As you point out especially in embedded systems development there are
segments where really really fast is critical and where fast enough is
often good enough. The tethered approach give you a rapid prototyping
platform for your target by using the host to implement your words.
That's a great idea. I'm struggling with the migration of those words to
the target. I'm trying to find a way out of what I perceive as a "gotta
compile the whole shebang" trap.
So the way I see it, even with a collection of optimized code words, in
order to gain the interactivity on the target I crave, I'd still have to
implement an inner interpreter to string the collection of optimized and
non optimzed words together, right?
Efficiency of execution isn't my primary goal. If I wanted to achieve
that then working on optimizing the PicForth compiler would be a better
use of my time. The tool I'm seeking is a fluid, interactive development
environment where at the end everything ends up on the target so I can
untether it and works fast enough to get the specified task done.
>> ...In my estimation one needs two sets of items to execute forth words:
>>
>> 1. The forth virtual machine including the standard registers and
>> stacks.
>>
>> 2. Three critical words: ENTER, EXIT, and NEXT.
>>
>> In short create enough Forth to execute the inner interpreter and you
>> can get going.
>Except that if you compile to code NEXT goes away, and ENTER/EXIT are
>merely subroutine call/return machine instructions.
But in my self-chosen constrained target environment this approach fails
on several levels given the goals I hope to achieve:
1. The pic's hardware stack is limited. Subroutine calls are simply not
an option because the stack overflows after only 8 levels of calls. This
is controllable in an assembly environment. But with Forth specifically
designed around making calls, it's a guaranteed path to doom.
2. Taking this route commits you to compiling your entire application
because once you do away with the inner interpreter, then everything on
the target must be compiled.
Let me outline how I envision using the target to give you a sense of
why I'm looking for a blended approach.
1. Starting out on a new project. Grab a part and use the traditional
programmer to dump the core executive on the part. Put traditional
programmer away until the next project because I absolutely detest
having to have a special programmer just to dump code on the chip.
Another advantage to Frank's kernel is that if it can write program
memory then it can serve as a bootloader for the chip even if I wanted
to dump something non Forth into it. I've tasked one of my summer interns
with writing a PIC16F bootloader in Forth combined with a picoforth
kernel that can program the PIC's program memory.
2. Wire up the project with the serial (or USB) interface and hook up to
the PC. Fire up gforth on the host and load the standard port
definitions and whatever words I have from previous projects that I
often use for embedded systems projects. Don't compile or download to
the target yet simply because I don't necessarily know what words I'll
actually need for the project.
3. As of now the target only has the microexecutive on it. But it's enough
to get started. I wire up whatever I/O I need for my application and
either test prewritten words on that I/O or write up new words necessary
to exercise it. All the debugging is done on the PC in gforth initially
until I'm happy with the result. I now start migration. When I get a
word I'm sure I'm going to need on the target, I move that word to the
target. Depending on the speed requirements, this may be a compiled word
which essentially functions as CODE, or a high level definition if speed
isn't critical. I retest the word on the target to make sure it works as
expected. Once that's done the word is added to the target wordset on
gforth and any further usage of that word will be remotely called.
4. Continue the process of building the application and incrementally
moving needed words to the target. Eventually the application will be
complete and well tested and all the words moved to the target and
nothing other than a GO command being run on the host.
5. Untether the target board, put it into service. Rinse and repeat with
the next project adding any interesting new words generated and tested
for this application to the hopefully growing library of useful words
that have been developed over previous projects.
Now in my view if an inner interpreter doesn't exist on the target that
activities 3 and 4 cannot be done. The inner interpreter is critical in
order to have both incremental compilation/movement of words to the
target and to facilitate the distributed execution of the application
between the host and target.
Did I miss something?
>> It seems to me that neither of those two items are so complicated that
>> they could not be easily put together for a target. And Frank's 3 word
>> forth implements a inner-inner interpreter that can be used to create
>> the inner interpreter required to run forth words on the target.
>Indisputably the traditional indirect-threaded model is easier for
>newbies to implement, but may not be satisfying for challenging or
>time-critical applications.
Most embedded application are not so time critical that it really
matters from a development standpoint. There's always a way to make it
faster. What I'm looking for is a way to make development easier.
This discussion reminds me of Brooks 90/10 rule I read in the Mythical
Man Month nearly 25 years ago. There's no need to have 100% of the code
optimized if only 10 percent of it is time critical. Local optimizations
are easy to do, simply compile (or write in assembly) critical words.
I addressed this issue when developing NPCI. Clearly running tokenized
bytecode isn't the fastest kid on the block. Speedup when necessary
could be achived by hooking optimized machine language code to the
interpreter then calling that code from the high level bytecode. It was
my equivalent of a CODE word, though FORTH specification mechanism is
vastly more elegant than my own.
Microcontrollers also have mechanisms for dealing with time critical
stuff. Another reason I love using PICs is the wide variety of hardware
periperals they come packaged. UARTS, multiple timers, PWM, ADC, and the
like are really set/autopilot types of tools. Interrupts can be used to
buffer really time sensitive stuff.
If all else fails after developing the application, simply run it all
through an optimizing compiler removing the inner interpreter altogether
along with other connecting tissue beween words.
There's always a faster crystal you can throw in. I'll probably test my
stuff at 8Mhz because PIC parts have that oscillator built in. If it's
not fast enough I'll throw a 20 Mhz crystal at it. If that's not fast
enough I'll get a propeller and now I have 8 20 MIPS processors to
handle the work. I can use the same technique to migrate the code there,
or simply run the finished product through Cliff's PropellerForth
compiler.
My time on a project is spent developing it. You (that would be
Elizabeth) pointed out in several posts over the years that developing
in a full fledged forth environment is a good thing. I agree. I firmly
believe that environment includes interactive and incremental
development. I'll sacrifice performance to get the project working.
"Make it work, then make it fast (only if necessary)".
>> Only one piece of the puzzle is missing at this point. Elizabeth
>> discusses in her post above that the XTL transfers the stack between the
>> host and the target. In short it implements a form of distributed
>> execution where you muster the stacks for RPC. In doing so one can run a
>> application with a set of words distributed between the host and the
>> target.
>>
>> So I envision an environment where Forth words are quick interactively
>> developed on the host. Once satisfied they work, they are incrementally
>> compiled on the host and transferred to the target. applications can be
>> a set of shared words between the two until complete at which point all
>> needed words are transferred to the target, completing the app.
>
>That tends not to be satisfactory, because most embedded apps feature
>custom I/O, so real target-specific words can't be tested on the host
>without your having to develop simulators for the I/O functions.
I don't think so. Franks microkernel is really just a remote memory
access. So if you have real I/O hardware wired to the target, then you
can write words on the host that accesses that hardware. It certainly
won't be full speed, but you can do it. Once it works in practice, you
then take the word, compile it, and transfer it to the target. In the
compilation you substitute direct memory access for remote memory
access. so Frank's XC@ compiles to @ and XC! compiles to !
Now you won't be able to gather a frame of video from a flash ADC this
way, but for testing it'll be useful.
>It's a
>lot easier to just go with a fully functional XTL and do all your
>testing on the target. Among other benefits, that means you can use
>Forth to debug your target hardware, which is wonderful.
What does a fully functional XTL offer? Right now it's kind of a
black box to me. Please enlighten me.
>> There are still details that need to be worked out such as how to
>> differentiate between local words and remote words in both systems and
>> how to facilitate transferring the stacks between the two. In both cases
>> solutions should be geared towards simplifying the target.
>
>That differentiation is typically done with wordsets (formerly known as
>vocabularies). The draft standard identifies "scopes" of words for the
>host, cross-compiler, and target; they are usually implemented with
>wordsets, although the draft standard doesn't mandate any particular
>implementation strategy.
I read that in one of the later chapters of Steven's book. Still a bit
fuzzy as to whether there's a concept of a local interpreter and a
remote interpreter though.
>> I do believe that the minimal kernel still needs to be specified. It'll
>> give a list of words that needs to be implemented, tested, and
>> incorporated in the initial kernel. I can help here because as I stated
>> earlier my NPCI bytecode interpreter implements a stack based virtual machine
>> and is written in pic assembly for the 16F family. It has debugged code
>> for all 16 bit arithmetic and logical ops, number processing for stacks,
>> and conditional/unconditional jumps. It already implements an IP for the
>> bytecode.
>
>Well, that sounds useful. Frankly, the reason so little Forth work has
>been done on the PIC family (and PICForth is so minimal) is that its
>instruction set is really a bit below the mark for a reasonable
>implementation. The PIC18 should be do-able, although we haven't yet
>seen much demand for it.
It's a chicken and egg problem. Anyone who has a real project with real
deadline will most likely either choose an existing development
environment for the target or choose a chip that is better supported by
Forth. I have the luxury of being an academic and a hobbyist. It also
helps to have a virtually unlimited supply of interns. So I can throw
resources at a project like this because it interests me, not because of
a deadline. There's of course a catch 22 to that too, which is that
since it isn't deadline driven development tends to be bursty.
>> I look forward to hearing your comments on these thoughts.
>
>I think it would be a good investment of your time to take a hard look
>at existing mature Forth cross-compilers. You can get a CD with
>extensive docs and links to free evaluation versions of our SwiftX
>cross-compilers for many chips (8051, 68HCS08, 68HC11, MSP430, AVR, ARM,
>68HC12, 68K family, Coldfire, more) for only $15. You can get supported
>boards for most of these processors very inexpensively. The evaluation
>compilers are limited only in the size of the target app you can
>develop, so you can exercise them and learn a lot. For more info go to
>http://www.forth.com/embedded/index.html.
I'll take a look. But frankly I won't get the warm fuzzies about it
until I'm sure that it in fact offers the type of environment I hoping
to run. It's also compilicating that SwiftX is a Windows product (and
justifiably so) and I'm a Linux guy (also justifiably so).
Thanks for the input. I'll take it under advisement and continue to
press on.
BAJ
foxchip
> > 2. Three critical words: ENTER, EXIT, and NEXT.
>
> > In short create enough Forth to execute the inner interpreter and you
> > can get going.
>
> Except that if you compile to code NEXT goes away, and ENTER/EXIT are
> merely subroutine call/return machine instructions.
In the case of macros that inline native code the ENTER/EXIT functions
can also be completely gone from the runtime code that gets compiled.
Best Wishes
Elizabeth D Rather
> In article <137quf9...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>> That said, improvements do occur. By hosting the actual compilation on
>> a powerful desktop, you can do things that are both difficult and
>> inappropriate on a limited target, such as compiling optimized target
>> machine code (which we do). The compile-to-optimized-machine-code
>> approach not only generates much faster code than the traditional
>> indirect-threaded approach, it's comparable in size on even small
>> targets such as the 8051, and significantly smaller on larger ones such
>> as the 68K family.
>
> I'm not necessarily opposed to the optimized-machine-code approach. It's
> just that with the PIC based forths I've seen so far, there's no
> incremental way to do it. So it reverts back to the traditional compile
> the whole app, download the whole app, test cycle that associated with
> traditional HLLs. And the target in this instance cannot be programmed
> at wire speed. So downloading a entire program to test takes a while.
> It's not like a PC where compiling is virtually instantaneous.
Well, we're able to do it incrementally on all the targets we support so
far. The actual compilation is on the host, but the download of the
result is immediate. We have a switch setting that either compiles the
whole thing and downloads it, or downloads individual definitions as
soon as they're done. We use the first mode to send the kernel, if
necessary, and then switch to the second mode for interactive development.
...
> As you point out especially in embedded systems development there are
> segments where really really fast is critical and where fast enough is
> often good enough. The tethered approach give you a rapid prototyping
> platform for your target by using the host to implement your words.
> That's a great idea. I'm struggling with the migration of those words to
> the target. I'm trying to find a way out of what I perceive as a "gotta
> compile the whole shebang" trap.
Well, your choice of the PIC16 clearly has some benefits, which you've
outlined, but it appears as though it's really handcuffing you in terms
of designing a workable development cycle.
> So the way I see it, even with a collection of optimized code words, in
> order to gain the interactivity on the target I crave, I'd still have to
> implement an inner interpreter to string the collection of optimized and
> non optimzed words together, right?
>
> Efficiency of execution isn't my primary goal. If I wanted to achieve
> that then working on optimizing the PicForth compiler would be a better
> use of my time. The tool I'm seeking is a fluid, interactive development
> environment where at the end everything ends up on the target so I can
> untether it and works fast enough to get the specified task done.
I don't see how the issue of incremental compilation and downloading is
dependent on the execution model. Regardless of what your compiled
stuff looks like (machine instructions, addresses, or tokens) you still
have to be able to download little bits of it and execute it, right?
Unless (I'm really pretty ignorant of PIC architecture) you have a
Harvard architecture and it's code space you have no access to. In that
case, either addresses or tokens could work (which is why we used tokens
on the AVR 8515).
...
>
> But in my self-chosen constrained target environment this approach fails
> on several levels given the goals I hope to achieve:
>
> 1. The pic's hardware stack is limited. Subroutine calls are simply not
> an option because the stack overflows after only 8 levels of calls. This
> is controllable in an assembly environment. But with Forth specifically
> designed around making calls, it's a guaranteed path to doom.
Well, it's somewhat limiting, but shouldn't be fatal. Most Forth apps
aren't really nested very deeply. We've run some pretty hairy apps on
8051's with on-chip stacks of limited size.
> 2. Taking this route commits you to compiling your entire application
> because once you do away with the inner interpreter, then everything on
> the target must be compiled.
No, it doesn't. Changing to direct code compilation didn't affect our
development cycle at all. Unless you're saying this based on another
PIC-specific obstacle we haven't heard about yet.
> Let me outline how I envision using the target to give you a sense of
> why I'm looking for a blended approach.
>
> 1. Starting out on a new project. Grab a part and use the traditional
> programmer to dump the core executive on the part. Put traditional
> programmer away until the next project because I absolutely detest
> having to have a special programmer just to dump code on the chip.
> Another advantage to Frank's kernel is that if it can write program
> memory then it can serve as a bootloader for the chip even if I wanted
> to dump something non Forth into it. I've tasked one of my summer interns
> with writing a PIC16F bootloader in Forth combined with a picoforth
> kernel that can program the PIC's program memory.
Yep, once your kernel (Frank's, ours, whatever) is downloaded, it should
be able to accept more stuff from the host provided your hardware gives
you access to a place to put it and run it.
> 2. Wire up the project with the serial (or USB) interface and hook up to
> the PC. Fire up gforth on the host and load the standard port
> definitions and whatever words I have from previous projects that I
> often use for embedded systems projects. Don't compile or download to
> the target yet simply because I don't necessarily know what words I'll
> actually need for the project.
That's possible assuming you really can make gForth look like your
target. Often there are issues. For example, gForth has a 32-bit cell
size, and your PIC model may find that too much overhead. If you're
running with 32-bit cells on your host and 16-bit cells on the target,
there may be some numbers that won't fit in 16 bits, so your gForth code
won't run on the target. That's just an example, but there are snakes
in those woods.
> 3. As of now the target only has the microexecutive on it. But it's enough
> to get started. I wire up whatever I/O I need for my application and
> either test prewritten words on that I/O or write up new words necessary
> to exercise it. All the debugging is done on the PC in gforth initially
> until I'm happy with the result. I now start migration. When I get a
> word I'm sure I'm going to need on the target, I move that word to the
> target. Depending on the speed requirements, this may be a compiled word
> which essentially functions as CODE, or a high level definition if speed
> isn't critical. I retest the word on the target to make sure it works as
> expected. Once that's done the word is added to the target wordset on
> gforth and any further usage of that word will be remotely called.
How does gForth access the I/O that's wired to your target? If you wire
it to the PC for testing, that's unlikely to be totally transparent.
We follow essentially this model, except all actual testing is done on
the target. This actually helps us debug the hardware interface as well
as the code.
> 4. Continue the process of building the application and incrementally
> moving needed words to the target. Eventually the application will be
> complete and well tested and all the words moved to the target and
> nothing other than a GO command being run on the host.
>
> 5. Untether the target board, put it into service. Rinse and repeat with
> the next project adding any interesting new words generated and tested
> for this application to the hopefully growing library of useful words
> that have been developed over previous projects.
Yes, that's certainly the preferred strategy.
> Now in my view if an inner interpreter doesn't exist on the target that
> activities 3 and 4 cannot be done. The inner interpreter is critical in
> order to have both incremental compilation/movement of words to the
> target and to facilitate the distributed execution of the application
> between the host and target.
>
> Did I miss something?
Yes. Whether there's an address interpreter (a term that's much more
descriptive than "inner" because it's clearer what's going on) or not
has no impact on the development cycle. A Harvard architecture part
requires somewhat different internal support for actual code vs. other
implementation strategies such as tokens or addresses, but the
development cycle can be made to look just the same.
...
> Microcontrollers also have mechanisms for dealing with time critical
> stuff. Another reason I love using PICs is the wide variety of hardware
> periperals they come packaged. UARTS, multiple timers, PWM, ADC, and the
> like are really set/autopilot types of tools. Interrupts can be used to
> buffer really time sensitive stuff.
So do most modern microcontrollers. Take another look at some of the
alternatives. There are some pretty nice parts out there.
> If all else fails after developing the application, simply run it all
> through an optimizing compiler removing the inner interpreter altogether
> along with other connecting tissue beween words.
That shouldn't have to be an extra step. An optimizing compiler isn't a
post-processor, it's an *alternative* to another kind of compiler (such
as ITC).
...
>
> My time on a project is spent developing it. You (that would be
> Elizabeth) pointed out in several posts over the years that developing
> in a full fledged forth environment is a good thing. I agree. I firmly
> believe that environment includes interactive and incremental
> development. I'll sacrifice performance to get the project working.
> "Make it work, then make it fast (only if necessary)".
What you seem to be doing is sacrificing the development cycle of your
dreams to stay with your beloved PICs. I think you'll find it really
hard to develop or support a good development system on the PIC16, from
all you've said.
>>> Only one piece of the puzzle is missing at this point. Elizabeth
>>> discusses in her post above that the XTL transfers the stack between the
>>> host and the target. In short it implements a form of distributed
>>> execution where you muster the stacks for RPC. In doing so one can run a
>>> application with a set of words distributed between the host and the
>>> target.
Well, it only models the data stack on the host. The return stack stays
on the target. And target words only execute on the target. There's no
attempt to simulate execution of target words on the host.
...
>> It's a
>> lot easier to just go with a fully functional XTL and do all your
>> testing on the target. Among other benefits, that means you can use
>> Forth to debug your target hardware, which is wonderful.
>
> What does a fully functional XTL offer? Right now it's kind of a
> black box to me. Please enlighten me.
The ability to have the "look and feel" of a full Forth on the target,
except that the actual target isn't burdened with dictionary heads &
searches, any kind of compiler or assembler, user interface, etc. All
these services are provided transparently by the host. I really urge
you to try SwiftX (or at least read its docs) to get a clearer picture.
>>> There are still details that need to be worked out such as how to
>>> differentiate between local words and remote words in both systems and
>>> how to facilitate transferring the stacks between the two. In both cases
>>> solutions should be geared towards simplifying the target.
>> That differentiation is typically done with wordsets (formerly known as
>> vocabularies). The draft standard identifies "scopes" of words for the
>> host, cross-compiler, and target; they are usually implemented with
>> wordsets, although the draft standard doesn't mandate any particular
>> implementation strategy.
>
> I read that in one of the later chapters of Steven's book. Still a bit
> fuzzy as to whether there's a concept of a local interpreter and a
> remote interpreter though.
I'm beginning to think you're confusing interpreters. A classic Forth
has a text interpreter, which processes text from a user or disk and
generates ("compiles") executable definitions (which might be actual
code, strings of addresses of words to be executed, tokens, or some
other internal form). That which used to be called an "inner"
interpreter, more accurately "address" interpreter, processes the
strings of addresses in the "compiled" form of a definition if that's
the model being used. It's usually only 1-3 machine instructions per
address, although on some processors it's more. In any case, I'll
repeat once again: the internal form of the definition has no impact on
the development cycle. These are orthogonal issues. There can be
excellent or terrible development tools with any internal Forth model.
Our systems have a text interpreter on the host, which parses your
command line or source file. Definitions are compiled (and whether the
compiled form is actual code, addresses, or tokens doesn't matter) and
downloaded to the host, either incrementally or in a batch depending on
switch setting. If you type a target command on the host, the host's
set of dictionary heads for the target is searched, and the target
address of the executable code is found. Then the target is directed to
execute it. The target does no interpreting.
...
>
> It's a chicken and egg problem. Anyone who has a real project with real
> deadline will most likely either choose an existing development
> environment for the target or choose a chip that is better supported by
> Forth. I have the luxury of being an academic and a hobbyist. It also
> helps to have a virtually unlimited supply of interns. So I can throw
> resources at a project like this because it interests me, not because of
> a deadline. There's of course a catch 22 to that too, which is that
> since it isn't deadline driven development tends to be bursty.
Well, the real issue is where you want to throw those resources: at tool
development or on the actual project. It's really easy to get
distracted into a lengthy tool design/development project instead of
actually working on the real one. Are your interns there to learn how
to write cross compilers, or do projects with microcontrollers?
>>> I look forward to hearing your comments on these thoughts.
>> I think it would be a good investment of your time to take a hard look
>> at existing mature Forth cross-compilers. You can get a CD with
>> extensive docs and links to free evaluation versions of our SwiftX
>> cross-compilers for many chips (8051, 68HCS08, 68HC11, MSP430, AVR, ARM,
>> 68HC12, 68K family, Coldfire, more) for only $15. You can get supported
>> boards for most of these processors very inexpensively. The evaluation
>> compilers are limited only in the size of the target app you can
>> develop, so you can exercise them and learn a lot. For more info go to
>> http://www.forth.com/embedded/index.html.
>
> I'll take a look. But frankly I won't get the warm fuzzies about it
> until I'm sure that it in fact offers the type of environment I hoping
> to run. It's also compilicating that SwiftX is a Windows product (and
> justifiably so) and I'm a Linux guy (also justifiably so).
Well, IMO the only way to find out if this is the type of environment
you're looking for is to try it. As for Windows vs. Linux, we don't
necessarily love Windows, but we need to make a living, and that's where
95% of the market is.
> Thanks for the input. I'll take it under advisement and continue to
> press on.
Enjoy,
dke...@hotmail.com
Hi
You should look into CMForth that was used on the NC4000. It had a
relatively simple cross compiler that is relatively easy to convert to
another platform. It is a little confusing at first because many low
level words exist as machine level words and are only binary codes.
Still, the entire FORTH is relatively easily recompiled. I've done
this many times myself to make changes or enhancements.
Dwight
Elizabeth D Rather
> In article <137quf9...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>> none Byron Jeff wrote:
>
> [Snippage]
>
>> There is at least a draft proposal for a cross-compiler addendum to ANS
>> Forth, at ftp://ftp.forth.com/pub/ANSForth. The actual proposed
>> standard is in XCtext5.doc, and non-normative explanatory appendices in
>> XCapp5.doc. There are also pdf forms. The '5' indicates that this is
>> the 5th draft, following an extended public review period.
>
>> (NOTE: at this moment, 9:30 am Hawaii time on 6/23, that link isn't
>> working. I'm trying to find out why and get it fixed. Meanwhile, if
>> you really want the docs now, email me erather {at} forth {dot} com)
>
> I'll take a look. I can wait.
The stuff has been moved, we're trying to get rid of ftp support. New
links are:
http://www.forth.com/downloads/ANS/XCtext5.doc (normative text,
http://www.forth.com/downloads/ANS/XCtext5.pdf two formats)
http://www.forth.com/downloads/ANS/XCapp5.doc (explanatory appendices,
http://www.forth.com/downloads/ANS/XCapp5.pdf two formats)
http://www.forth.com/downloads/ANS/XCpaper.pdf (overview)
XCpaper.pdf is an overview of the concepts, and a good place to start.
none Byron Jeff
OK. So we're getting somewhere. Now I happen to still be stuck due to
the unreasonablly small hardware stack size for my particular target.
I presume that the optimized compiled code is based on STC with
optimizations?
>...
>> As you point out especially in embedded systems development there are
>> segments where really really fast is critical and where fast enough is
>> often good enough. The tethered approach give you a rapid prototyping
>> platform for your target by using the host to implement your words.
>> That's a great idea. I'm struggling with the migration of those words to
>> the target. I'm trying to find a way out of what I perceive as a "gotta
>> compile the whole shebang" trap.
>
>Well, your choice of the PIC16 clearly has some benefits, which you've
>outlined, but it appears as though it's really handcuffing you in terms
>of designing a workable development cycle.
I don't think so. Execution effiency isn't my primary goal, and that's
what will be handcuffed. I was working with a bytecode interpreter that
is probably averaging 40 instructions per bytecode on average. At the
time I started working on it years ago, my bytecode memory was a
bitbanged serial EEPROM. Execution effiency isn't a worry. It's the item
I can most afford to give up initially.
>> So the way I see it, even with a collection of optimized code words, in
>> order to gain the interactivity on the target I crave, I'd still have to
>> implement an inner interpreter to string the collection of optimized and
>> non optimzed words together, right?
>>
>> Efficiency of execution isn't my primary goal. If I wanted to achieve
>> that then working on optimizing the PicForth compiler would be a better
>> use of my time. The tool I'm seeking is a fluid, interactive development
>> environment where at the end everything ends up on the target so I can
>> untether it and works fast enough to get the specified task done.
>
>I don't see how the issue of incremental compilation and downloading is
>dependent on the execution model.
>Regardless of what your compiled
>stuff looks like (machine instructions, addresses, or tokens) you still
>have to be able to download little bits of it and execute it, right?
The little bits is the key. PicForth and Mary are organized as a one
shot compilation environment. You compile the entire system and download
it at one go. I want pretty much the opposite. The current tools I have
available to me are not organized that way.
>Unless (I'm really pretty ignorant of PIC architecture) you have a
>Harvard architecture and it's code space you have no access to.
It's a Harvard architecture that's difficult to access and slow to
update. So transferring little bits at a time is a highly desireable
trait.
>In that
>case, either addresses or tokens could work (which is why we used tokens
>on the AVR 8515).
Tokens is the winner in my case too. Still a bit concerned about if I'm
constrained in my token size (or if it really matters).
>
>...
>>
>> But in my self-chosen constrained target environment this approach fails
>> on several levels given the goals I hope to achieve:
>>
>> 1. The pic's hardware stack is limited. Subroutine calls are simply not
>> an option because the stack overflows after only 8 levels of calls. This
>> is controllable in an assembly environment. But with Forth specifically
>> designed around making calls, it's a guaranteed path to doom.
>
>Well, it's somewhat limiting, but shouldn't be fatal. Most Forth apps
>aren't really nested very deeply. We've run some pretty hairy apps on
>8051's with on-chip stacks of limited size.
That's encouraging. I just owrry about reliability because if you
overflow the hardware stack, your application is guaranteed to crash eventually.
And that stack is completely unmapped in memory. It's probably the thing
about the part that drives me the most crazy because you can't implement
any effective context switching without access to that stack.
Exactly how limited were those 8051 stacks?
>> 2. Taking this route commits you to compiling your entire application
>> because once you do away with the inner interpreter, then everything on
>> the target must be compiled.
>
>No, it doesn't. Changing to direct code compilation didn't affect our
>development cycle at all. Unless you're saying this based on another
>PIC-specific obstacle we haven't heard about yet.
Not sure. I think I'm reaching the boundaries of my understanding. If
point #1 above is taken off the table, then I think I can see it because
implementing optimized STC eliminates the interpreter yet facilitates
incremental additions to the codebase on the target. But if that
hardware stack is out of bounds, I'm lost as to how you could implement
ITC, DTC, or TTC without elements of the address interpreter.
>> Let me outline how I envision using the target to give you a sense of
>> why I'm looking for a blended approach.
>>
>> 1. Starting out on a new project. Grab a part and use the traditional
>> programmer to dump the core executive on the part. Put traditional
>> programmer away until the next project because I absolutely detest
>> having to have a special programmer just to dump code on the chip.
>> Another advantage to Frank's kernel is that if it can write program
>> memory then it can serve as a bootloader for the chip even if I wanted
>> to dump something non Forth into it. I've tasked one of my summer interns
>> with writing a PIC16F bootloader in Forth combined with a picoforth
>> kernel that can program the PIC's program memory.
>Yep, once your kernel (Frank's, ours, whatever) is downloaded, it should
>be able to accept more stuff from the host provided your hardware gives
>you access to a place to put it and run it.
That's a good start.
>> 2. Wire up the project with the serial (or USB) interface and hook up to
>> the PC. Fire up gforth on the host and load the standard port
>> definitions and whatever words I have from previous projects that I
>> often use for embedded systems projects. Don't compile or download to
>> the target yet simply because I don't necessarily know what words I'll
>> actually need for the project.
>That's possible assuming you really can make gForth look like your
>target. Often there are issues. For example, gForth has a 32-bit cell
>size, and your PIC model may find that too much overhead. If you're
>running with 32-bit cells on your host and 16-bit cells on the target,
>there may be some numbers that won't fit in 16 bits, so your gForth code
>won't run on the target. That's just an example, but there are snakes
>in those woods.
Worth rooting them out because I can get instant gratification using
gforth to develop.
>> 3. As of now the target only has the microexecutive on it. But it's enough
>> to get started. I wire up whatever I/O I need for my application and
>> either test prewritten words on that I/O or write up new words necessary
>> to exercise it. All the debugging is done on the PC in gforth initially
>> until I'm happy with the result. I now start migration. When I get a
>> word I'm sure I'm going to need on the target, I move that word to the
>> target. Depending on the speed requirements, this may be a compiled word
>> which essentially functions as CODE, or a high level definition if speed
>> isn't critical. I retest the word on the target to make sure it works as
>> expected. Once that's done the word is added to the target wordset on
>> gforth and any further usage of that word will be remotely called.
>How does gForth access the I/O that's wired to your target? If you wire
>it to the PC for testing, that's unlikely to be totally transparent.
I/O is memory accessed. Remote access is via Frank's 3 instruction
implementation. Local access is via @ and !. The word compiler to the
target that runs under gforth can transform remote memory accesses into
local one. So it'll be transparent to the developer when the code is
migrated to the target.
>We follow essentially this model, except all actual testing is done on
>the target. This actually helps us debug the hardware interface as well
>as the code.
But it gets back to the question of how much kernel do you have to
download to get started? At the very least in bootstrapping the kernel a
distributed model would have to be in play.
>> 4. Continue the process of building the application and incrementally
>> moving needed words to the target. Eventually the application will be
>> complete and well tested and all the words moved to the target and
>> nothing other than a GO command being run on the host.
>>
>> 5. Untether the target board, put it into service. Rinse and repeat with
>> the next project adding any interesting new words generated and tested
>> for this application to the hopefully growing library of useful words
>> that have been developed over previous projects.
>
>Yes, that's certainly the preferred strategy.
>
>> Now in my view if an inner interpreter doesn't exist on the target that
>> activities 3 and 4 cannot be done. The inner interpreter is critical in
>> order to have both incremental compilation/movement of words to the
>> target and to facilitate the distributed execution of the application
>> between the host and target.
>>
>> Did I miss something?
>
>Yes. Whether there's an address interpreter (a term that's much more
>descriptive than "inner" because it's clearer what's going on)
Will switch...
>or not
>has no impact on the development cycle. A Harvard architecture part
>requires somewhat different internal support for actual code vs. other
>implementation strategies such as tokens or addresses, but the
>development cycle can be made to look just the same.
I guess my question is what is the structure of an optimized compiled
code word then? What I cannot visualize is the linkages between the code
fragments.
I think that structurally I can easily see how to compile a definition
into a collection of addresses or tokens. However compiling native code
is a different animal.
>
>...
>> Microcontrollers also have mechanisms for dealing with time critical
>> stuff. Another reason I love using PICs is the wide variety of hardware
>> periperals they come packaged. UARTS, multiple timers, PWM, ADC, and the
>> like are really set/autopilot types of tools. Interrupts can be used to
>> buffer really time sensitive stuff.
>
>So do most modern microcontrollers. Take another look at some of the
>alternatives. There are some pretty nice parts out there.
I'm aware. Remember I got here because I was looking at the propeller.
I'm already having to get up to speed with a new language and a new
tool. Componding it by starting from scratch with a new architecture is
too much to tackle.
Plus I feel if I can pull this off in my constrained little box, that
moving the port to a roomier chip (like the propeller, which of course
due to Cliff I don't need to do) should be no problem.
>> If all else fails after developing the application, simply run it all
>> through an optimizing compiler removing the inner interpreter altogether
>> along with other connecting tissue beween words.
>
>That shouldn't have to be an extra step. An optimizing compiler isn't a
>post-processor, it's an *alternative* to another kind of compiler (such
>as ITC).
An incremental optimizing Forth compiler for the PIC 16F platform
doesn't exist AFAICT. It needs to be built. My experience with language
tool building and with pics tells me that the optimizing compiler is the
much tougher road to travel to get to a incremental development target.
A non incremental optimizing compiler does exist. But I doesn't suit my
development needs.
Put the two together and the answer that pops out is to implement a non
optimized token based compiler. I'm in my wheelhouse there because I
already have a token based, stack implemented 16F kernel that's already
tested and can be quick adapted to the task.
While I do have fun building tools, they do have the purpose of building
other stuff. I prefer building the simplest foolproof tools I can build
then using them to bootstrap up.
Implementing an address interpreter with NEXT, ENTER, and EXIT "words"
will cost me an afternoon and about 25-30 lines of assembly. Then I'll
have a tool that I can use to put forth on my target.
I'm not worried about slow. I'm worring about getting done and having
the right result when I get done.
>> My time on a project is spent developing it. You (that would be
>> Elizabeth) pointed out in several posts over the years that developing
>> in a full fledged forth environment is a good thing. I agree. I firmly
>> believe that environment includes interactive and incremental
>> development. I'll sacrifice performance to get the project working.
>> "Make it work, then make it fast (only if necessary)".
>
>What you seem to be doing is sacrificing the development cycle of your
>dreams to stay with your beloved PICs. I think you'll find it really
>hard to develop or support a good development system on the PIC16, from
>all you've said.
Every other development cycle has its costs too. There's the cost of
learning new architectures, the cost of new programming tools and
software, the cost of sacrificing prototypability. For example the TI
MPS430 only comes in 3.3V or less version, with no 5V tolerant I/O and
in quad flat pack packaging only. Major shift.
And PICs are not the only tool in my box. Being a Linux guy (and yes
that's actually non negotiable) means I have to be extremely picky and
choosy about the tools I put in my box.
But I have the luxury of doing so as an acadmic and hobbyist. No
constraints, project deadlines, sales targets projections, or budget
concerns to worry about.
>>>> Only one piece of the puzzle is missing at this point. Elizabeth
>>>> discusses in her post above that the XTL transfers the stack between the
>>>> host and the target. In short it implements a form of distributed
>>>> execution where you muster the stacks for RPC. In doing so one can run a
>>>> application with a set of words distributed between the host and the
>>>> target.
>
>Well, it only models the data stack on the host. The return stack stays
>on the target. And target words only execute on the target. There's no
>attempt to simulate execution of target words on the host.
Ah. I see. So that means that your XTL had to be significantly developed
before you could start using it. The appeal of Frank's paper was that
essentially once you implemented his three instructions kernel, that you
could immediately start developing applications with it without needing
to flesh out an entire kernel just to get started. This leads back to
the point I made in my initial post that a good (albeit slow) small set
of primitives would be good to implement. And the 48 that I've seen for
MAF doesn't qualify as a small set.
I see the distributed model sort of as a breakpoint. The host already
has everything (primitives, core words, core extensions) already
implemented. Why not use it as a remote process server in addition to
the text interpreter, wordlist coordinator, and the target's cross
compiler?
>
>...
>>> It's a
>>> lot easier to just go with a fully functional XTL and do all your
>>> testing on the target. Among other benefits, that means you can use
>>> Forth to debug your target hardware, which is wonderful.
>>
>> What does a fully functional XTL offer? Right now it's kind of a
>> black box to me. Please enlighten me.
>
>The ability to have the "look and feel" of a full Forth on the target,
>except that the actual target isn't burdened with dictionary heads &
>searches, any kind of compiler or assembler, user interface, etc. All
>these services are provided transparently by the host. I really urge
>you to try SwiftX (or at least read its docs) to get a clearer picture.
I may take a read of the docs.
>>>> There are still details that need to be worked out such as how to
>>>> differentiate between local words and remote words in both systems and
>>>> how to facilitate transferring the stacks between the two. In both cases
>>>> solutions should be geared towards simplifying the target.
>>> That differentiation is typically done with wordsets (formerly known as
>>> vocabularies). The draft standard identifies "scopes" of words for the
>>> host, cross-compiler, and target; they are usually implemented with
>>> wordsets, although the draft standard doesn't mandate any particular
>>> implementation strategy.
>>
>> I read that in one of the later chapters of Steven's book. Still a bit
>> fuzzy as to whether there's a concept of a local interpreter and a
>> remote interpreter though.
>
>I'm beginning to think you're confusing interpreters.
Nope. I have it straight.
>A classic Forth
>has a text interpreter, which processes text from a user or disk and
>generates ("compiles") executable definitions (which might be actual
>code, strings of addresses of words to be executed, tokens, or some
>other internal form). That which used to be called an "inner"
>interpreter, more accurately "address" interpreter, processes the
>strings of addresses in the "compiled" form of a definition if that's
>the model being used. It's usually only 1-3 machine instructions per
>address, although on some processors it's more.
Right. The point of running Forth on the host is to get a complete
environment without having the burden the target with it. This is the
tethered model. But no one seems to be addressing the possibility of
distributed computing between the host and the target. The host is
simply a respository for a set of services (text interpreter, cross
compiler, wordset dictionaries) without helping the target run any
actual forth code. The way I see it since the host is a full forth
environment, it can emulate a full forth environment for the target.
>In any case, I'll
>repeat once again: the internal form of the definition has no impact on
>the development cycle. These are orthogonal issues. There can be
>excellent or terrible development tools with any internal Forth model.
I believe that now. You've broken the connection between optimized
native code definitions and linkage technique in my mind. Thanks for
that.
So given that how does one go about building a optimized code compiler
that functions in an incremental fashion for a target that doesn't yet
have such a beast?
>Our systems have a text interpreter on the host, which parses your
>command line or source file. Definitions are compiled (and whether the
>compiled form is actual code, addresses, or tokens doesn't matter) and
>downloaded to the host, either incrementally or in a batch depending on
>switch setting. If you type a target command on the host, the host's
>set of dictionary heads for the target is searched, and the target
>address of the executable code is found. Then the target is directed to
>execute it. The target does no interpreting.
But you can't have it both ways. I'm not talking about text
interpretation at all. Only address interpretation. Unless I missed
something the only two ways to compile definitions without an address
interpreter are STC or by inlining the code. If the compiled form is
addresses or tokens, then the target by definition needs to have an
address interpreter to interpret those addresses or tokens.
The beauty though is that the address interpreter words are the only
words required to execute forth words on the target presuming that those
definitions are compiled into addresses or tokens. And as you implied
above, host compilation in that instance is nothing more than looking up
those addresses/tokens in the dictionary heads and emitting a collection of
tokens or addresses corresponding to what is found in the dictionary.
BTW I realized that I'm still trying to figure out how in the heck forth
compiles a number into a definition. What is the xt for a number?
>
>...
>>
>> It's a chicken and egg problem. Anyone who has a real project with real
>> deadline will most likely either choose an existing development
>> environment for the target or choose a chip that is better supported by
>> Forth. I have the luxury of being an academic and a hobbyist. It also
>> helps to have a virtually unlimited supply of interns. So I can throw
>> resources at a project like this because it interests me, not because of
>> a deadline. There's of course a catch 22 to that too, which is that
>> since it isn't deadline driven development tends to be bursty.
>
>Well, the real issue is where you want to throw those resources: at tool
>development or on the actual project. It's really easy to get
>distracted into a lengthy tool design/development project instead of
>actually working on the real one. Are your interns there to learn how
>to write cross compilers, or do projects with microcontrollers?
Both. They are not doing any of this tool work because it's
still getting specified here in this thread. They'll use picforth to
compile their applications. But they'll be stuck in the edit, compile,
download, test cycle because no other application environment currently
exists for them to do anything else.
That's why I'm here having this discussion.
>
>>>> I look forward to hearing your comments on these thoughts.
>>> I think it would be a good investment of your time to take a hard look
>>> at existing mature Forth cross-compilers. You can get a CD with
>>> extensive docs and links to free evaluation versions of our SwiftX
>>> cross-compilers for many chips (8051, 68HCS08, 68HC11, MSP430, AVR, ARM,
>>> 68HC12, 68K family, Coldfire, more) for only $15. You can get supported
>>> boards for most of these processors very inexpensively. The evaluation
>>> compilers are limited only in the size of the target app you can
>>> develop, so you can exercise them and learn a lot. For more info go to
>>> http://www.forth.com/embedded/index.html.
>>
>> I'll take a look. But frankly I won't get the warm fuzzies about it
>> until I'm sure that it in fact offers the type of environment I hoping
>> to run. It's also compilicating that SwiftX is a Windows product (and
>> justifiably so) and I'm a Linux guy (also justifiably so).
>
>Well, IMO the only way to find out if this is the type of environment
>you're looking for is to try it. As for Windows vs. Linux, we don't
>necessarily love Windows, but we need to make a living, and that's where
>95% of the market is.
I know. That's why I said justifiably so. My small aside on that subject
is that if tool developers could find a way to develop cross platform
tools without expending too much additional effort, then maybe a more
equitable distribution of market share would follow.
BAJ
Elizabeth D Rather
> In article <137rks1...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>> none Byron Jeff wrote:
>>> I'm not necessarily opposed to the optimized-machine-code approach. It's
>>> just that with the PIC based forths I've seen so far, there's no
>>> incremental way to do it. So it reverts back to the traditional compile
>>> the whole app, download the whole app, test cycle that associated with
>>> traditional HLLs. And the target in this instance cannot be programmed
>>> at wire speed. So downloading a entire program to test takes a while.
>>> It's not like a PC where compiling is virtually instantaneous.
>> Well, we're able to do it incrementally on all the targets we support so
>> far. The actual compilation is on the host, but the download of the
>> result is immediate. We have a switch setting that either compiles the
>> whole thing and downloads it, or downloads individual definitions as
>> soon as they're done. We use the first mode to send the kernel, if
>> necessary, and then switch to the second mode for interactive development.
>
> OK. So we're getting somewhere. Now I happen to still be stuck due to
> the unreasonablly small hardware stack size for my particular target.
> I presume that the optimized compiled code is based on STC with
> optimizations?
Yes, but it doesn't matter. The point I'm trying to make is that the
choice of implementation model doesn't preclude incremental compilation.
...
>> Well, your choice of the PIC16 clearly has some benefits, which you've
>> outlined, but it appears as though it's really handcuffing you in terms
>> of designing a workable development cycle.
>
> I don't think so. Execution effiency isn't my primary goal, and that's
> what will be handcuffed. I was working with a bytecode interpreter that
> is probably averaging 40 instructions per bytecode on average. At the
> time I started working on it years ago, my bytecode memory was a
> bitbanged serial EEPROM. Execution effiency isn't a worry. It's the item
> I can most afford to give up initially.
You are handcuffed in the sense that you would like to be able to
download small amounts of code into ram and execute it. You don't seem
to have enough ram to do this, not to mention not enough stack space,
etc. Hence, a civilized development environment will be very much more
difficult to arrange than on other platforms.
...
>> I don't see how the issue of incremental compilation and downloading is
>> dependent on the execution model.
>> Regardless of what your compiled
>> stuff looks like (machine instructions, addresses, or tokens) you still
>> have to be able to download little bits of it and execute it, right?
>
> The little bits is the key. PicForth and Mary are organized as a one
> shot compilation environment. You compile the entire system and download
> it at one go. I want pretty much the opposite. The current tools I have
> available to me are not organized that way.
Right. But the problem is that the tools are designed for the
limitations of the platform. A less limited platform can more easily
support the kind of tools you're seeking.
>> Unless (I'm really pretty ignorant of PIC architecture) you have a
>> Harvard architecture and it's code space you have no access to.
>
> It's a Harvard architecture that's difficult to access and slow to
> update. So transferring little bits at a time is a highly desireable
> trait.
Yes. But you need somewhere to transfer them to that's a little more
accessible than that. PIC doesn't seem to support ram development,
which is the best way to do incremental testing.
>> In that
>> case, either addresses or tokens could work (which is why we used tokens
>> on the AVR 8515).
>
> Tokens is the winner in my case too. Still a bit concerned about if I'm
> constrained in my token size (or if it really matters).
Not really.
>> ...
>>> But in my self-chosen constrained target environment this approach fails
>>> on several levels given the goals I hope to achieve:
>>>
>>> 1. The pic's hardware stack is limited. Subroutine calls are simply not
>>> an option because the stack overflows after only 8 levels of calls. This
>>> is controllable in an assembly environment. But with Forth specifically
>>> designed around making calls, it's a guaranteed path to doom.
>> Well, it's somewhat limiting, but shouldn't be fatal. Most Forth apps
>> aren't really nested very deeply. We've run some pretty hairy apps on
>> 8051's with on-chip stacks of limited size.
>
> That's encouraging. I just owrry about reliability because if you
> overflow the hardware stack, your application is guaranteed to crash eventually.
> And that stack is completely unmapped in memory. It's probably the thing
> about the part that drives me the most crazy because you can't implement
> any effective context switching without access to that stack.
Among the many reasons we avoid using PICs.
> Exactly how limited were those 8051 stacks?
64 bytes (32 cells) as I recall. Could have been 48 bytes. It's been a
while.
>>> 2. Taking this route commits you to compiling your entire application
>>> because once you do away with the inner interpreter, then everything on
>>> the target must be compiled.
>> No, it doesn't. Changing to direct code compilation didn't affect our
>> development cycle at all. Unless you're saying this based on another
>> PIC-specific obstacle we haven't heard about yet.
>
> Not sure. I think I'm reaching the boundaries of my understanding. If
> point #1 above is taken off the table, then I think I can see it because
> implementing optimized STC eliminates the interpreter yet facilitates
> incremental additions to the codebase on the target. But if that
> hardware stack is out of bounds, I'm lost as to how you could implement
> ITC, DTC, or TTC without elements of the address interpreter.
The execution model doesn't affect whether you can or cannot do
incremental compilation. What determines that is whether you have a
place to put the downloaded definitions to test them. You make it sound
like the address interpreter is a big deal. It isn't.
...
> But it gets back to the question of how much kernel do you have to
> download to get started? At the very least in bootstrapping the kernel a
> distributed model would have to be in play.
You start with a couple dozen bytes and use that to load the rest. We
find a few K to be capable of providing a useful set of primitives.
...
>
> I guess my question is what is the structure of an optimized compiled
> code word then? What I cannot visualize is the linkages between the code
> fragments.
It's code in code space. Small primitives or optimized code sequences
are expanded in place. Larger words are called. Linkage is normal
call/return.
> I think that structurally I can easily see how to compile a definition
> into a collection of addresses or tokens. However compiling native code
> is a different animal.
Simpler, really.
>> ...
>>> Microcontrollers also have mechanisms for dealing with time critical
>>> stuff. Another reason I love using PICs is the wide variety of hardware
>>> periperals they come packaged. UARTS, multiple timers, PWM, ADC, and the
>>> like are really set/autopilot types of tools. Interrupts can be used to
>>> buffer really time sensitive stuff.
>> So do most modern microcontrollers. Take another look at some of the
>> alternatives. There are some pretty nice parts out there.
>
> I'm aware. Remember I got here because I was looking at the propeller.
> I'm already having to get up to speed with a new language and a new
> tool. Componding it by starting from scratch with a new architecture is
> too much to tackle.
It's a lot easier to come up to speed with a new language if you have
thoroughly tested, well-documented tools at hand. Trying to develop
tools for a language that's new to you is a much worse challenge.
> Plus I feel if I can pull this off in my constrained little box, that
> moving the port to a roomier chip (like the propeller, which of course
> due to Cliff I don't need to do) should be no problem.
Isn't that like saying, if I can learn to ride a unicycle, a tricycle
will be easy?
>>> If all else fails after developing the application, simply run it all
>>> through an optimizing compiler removing the inner interpreter altogether
>>> along with other connecting tissue beween words.
>> That shouldn't have to be an extra step. An optimizing compiler isn't a
>> post-processor, it's an *alternative* to another kind of compiler (such
>> as ITC).
>
> An incremental optimizing Forth compiler for the PIC 16F platform
> doesn't exist AFAICT. It needs to be built. My experience with language
> tool building and with pics tells me that the optimizing compiler is the
> much tougher road to travel to get to a incremental development target.
True, but even developing an interactive, incremental ITC or token-based
development system for this beast will be extremely difficult.
> A non incremental optimizing compiler does exist. But I doesn't suit my
> development needs.
It was developed by a very clever guy. I'm sure if he could have built
an incremental, interactive compiler he would have.
> Put the two together and the answer that pops out is to implement a non
> optimized token based compiler. I'm in my wheelhouse there because I
> already have a token based, stack implemented 16F kernel that's already
> tested and can be quick adapted to the task.
>
> While I do have fun building tools, they do have the purpose of building
> other stuff. I prefer building the simplest foolproof tools I can build
> then using them to bootstrap up.
>
> Implementing an address interpreter with NEXT, ENTER, and EXIT "words"
> will cost me an afternoon and about 25-30 lines of assembly. Then I'll
> have a tool that I can use to put forth on my target.
>
> I'm not worried about slow. I'm worring about getting done and having
> the right result when I get done.
Well, it sounds as though you have months of fun ahead of you.
...
>>>>> Only one piece of the puzzle is missing at this point. Elizabeth
>>>>> discusses in her post above that the XTL transfers the stack between the
>>>>> host and the target. In short it implements a form of distributed
>>>>> execution where you muster the stacks for RPC. In doing so one can run a
>>>>> application with a set of words distributed between the host and the
>>>>> target.
>> Well, it only models the data stack on the host. The return stack stays
>> on the target. And target words only execute on the target. There's no
>> attempt to simulate execution of target words on the host.
>
> Ah. I see. So that means that your XTL had to be significantly developed
> before you could start using it. The appeal of Frank's paper was that
> essentially once you implemented his three instructions kernel, that you
> could immediately start developing applications with it without needing
> to flesh out an entire kernel just to get started. This leads back to
> the point I made in my initial post that a good (albeit slow) small set
> of primitives would be good to implement. And the 48 that I've seen for
> MAF doesn't qualify as a small set.
The target side of the XTL is very small and simple, probably not
significantly different from Frank's concept. The host is rather more
complex. And the concepts have been developed over about 20 years.
There's a lot of advantages to be found in "standing on the shoulders of
giants".
> I see the distributed model sort of as a breakpoint. The host already
> has everything (primitives, core words, core extensions) already
> implemented. Why not use it as a remote process server in addition to
> the text interpreter, wordlist coordinator, and the target's cross
> compiler?
Yes.
...
>> A classic Forth
>> has a text interpreter, which processes text from a user or disk and
>> generates ("compiles") executable definitions (which might be actual
>> code, strings of addresses of words to be executed, tokens, or some
>> other internal form). That which used to be called an "inner"
>> interpreter, more accurately "address" interpreter, processes the
>> strings of addresses in the "compiled" form of a definition if that's
>> the model being used. It's usually only 1-3 machine instructions per
>> address, although on some processors it's more.
>
> Right. The point of running Forth on the host is to get a complete
> environment without having the burden the target with it. This is the
> tethered model. But no one seems to be addressing the possibility of
> distributed computing between the host and the target. The host is
> simply a respository for a set of services (text interpreter, cross
> compiler, wordset dictionaries) without helping the target run any
> actual forth code. The way I see it since the host is a full forth
> environment, it can emulate a full forth environment for the target.
Yes, the host provides all those services. What it doesn't do well is
execute target code. Therefore, we make no attempt to execute target
code on the host, but transparently exercise it on the target.
>> In any case, I'll
>> repeat once again: the internal form of the definition has no impact on
>> the development cycle. These are orthogonal issues. There can be
>> excellent or terrible development tools with any internal Forth model.
>
> I believe that now. You've broken the connection between optimized
> native code definitions and linkage technique in my mind. Thanks for
> that.
Good.
> So given that how does one go about building a optimized code compiler
> that functions in an incremental fashion for a target that doesn't yet
> have such a beast?
Don't bother for now. A token-based implementation should work fine, so
long as you're stuck with this PIC.
>> Our systems have a text interpreter on the host, which parses your
>> command line or source file. Definitions are compiled (and whether the
>> compiled form is actual code, addresses, or tokens doesn't matter) and
>> downloaded to the host, either incrementally or in a batch depending on
>> switch setting. If you type a target command on the host, the host's
>> set of dictionary heads for the target is searched, and the target
>> address of the executable code is found. Then the target is directed to
>> execute it. The target does no interpreting.
>
> But you can't have it both ways. I'm not talking about text
> interpretation at all. Only address interpretation. Unless I missed
> something the only two ways to compile definitions without an address
> interpreter are STC or by inlining the code. If the compiled form is
> addresses or tokens, then the target by definition needs to have an
> address interpreter to interpret those addresses or tokens.
Yes. But that has no impact on the development style.
...
>
> BTW I realized that I'm still trying to figure out how in the heck forth
> compiles a number into a definition. What is the xt for a number?
It's the address for (or call to) a word that pushes the actual value on
the stack, followed by the value:
... [xt of LIT] [value] ...
There may be different versions for 8, 16, or 32-bit literals. It has
to advance the interpreter pointer or PC beyond the value, in addition
to pushing the value on the stack.
...
>>> I'll take a look. But frankly I won't get the warm fuzzies about it
>>> until I'm sure that it in fact offers the type of environment I hoping
>>> to run. It's also complicating that SwiftX is a Windows product (and
>>> justifiably so) and I'm a Linux guy (also justifiably so).
>> Well, IMO the only way to find out if this is the type of environment
>> you're looking for is to try it. As for Windows vs. Linux, we don't
>> necessarily love Windows, but we need to make a living, and that's where
>> 95% of the market is.
>
> I know. That's why I said justifiably so. My small aside on that subject
> is that if tool developers could find a way to develop cross platform
> tools without expending too much additional effort, then maybe a more
> equitable distribution of market share would follow.
It's not just that the platforms are different. Windows users have
certain expectations of their development system (e.g. pull-down menus,
toolbars, much more) that a simple command-line Forth like gForth
doesn't support. If you don't provide them, your system won't sell and
all your effort is wasted. And actually, they're pretty nifty. Since
we use our tools to develop very complex applications, we are
continually improving them to make them easier to use.
Cheers,
none Byron Jeff
> You should look into CMForth that was used on the NC4000. It had a
>relatively simple cross compiler that is relatively easy to convert to
>another platform. It is a little confusing at first because many low
>level words exist as machine level words and are only binary codes.
That means that it isn't necessarily minimal. Maybe I should stop
complaining because I have my own bytecode interpreter that has about 35
instructions. I factored the binary ops by writing a common routine that
pops the stack into a temp register (binopsetup). So most of those
instructions were little more than doing a binopsetup followed by the
actual task of the instruction, then writing back the result to the new
TOS. That accounted for about half the group
(+,-,=,!=,>,<,>=,<=,&,|,^,&&,||,<<,>>)
An interesting question is how many of these could you actually factor?
All 6 comparison operators are buildable from a subtract, as is addition
if you throw in ^. AND an OR are negative complements of one another so
you really only need NAND to build both (and ^ too IIRC). && and || are
buildable from comparison and logical ops. Left shift can be done via
recursive addition. Not sure about right shift.
So the kernel for the 15 above tentatively is: -, NAND, >>
You then throw in the equivalents for ! and @, unary ops, pushing
numbers on the stack, conditional and unconditional goto, and you're
pushing 20. Since I was targeting a traditional language I didn't throw
in but a couple of stack manipulation operators. I guess implementing
DROP, SWAP, DUP, OVER would be good. Any others?
The one mistake I made was encoding addresses into bytecodes instead of
pushing them onto the stack just like everything else. I beleve that's
the Forth tick ' operator right?
Finally since this is targeting an embedded system environment having
bit manipulation is also helpful. Fundamentally implementing b! and b@
>Still, the entire FORTH is relatively easily recompiled. I've done
>this many times myself to make changes or enhancements.
After last night's discussion with Elizabeth, I'm still trying to wrap
my head around exactly what does "compile" mean.
BAJ
Stephen Pelc
wrote:
Elizabeth has explained enough of the mechanics that I won't repeat
here explanations. By the way, I'm a Stephen not a Steven.
The problem with PICs for Forth is that the early ones are based
on a 1970s architecture for a minicomputer I/O processor. The
benefits of PICs are in the price and the peripherals. An
undoubted benefit for hobbyists is that you can still get them
in DIP packages.
For commercial vendors of Forth, the price point of the
standard tools does not make PICs a very attractive target,
especially as high-end PICs are more expensive than low-end
ARMs. The PIC18 was the first Forth-friendly PIC, but these
days has little to offer unless you are already a PIC user.
Within MPE, the question of supplying a PIC system is raised
every year or so, and we always end up saying "not unless
a client sponsors it".
>So the question is "Given an embedded micro, what is the minimum forth
>environment required to execute Forth words?".
It has been done many times using: fetch, store, execute. For a
Harvard architecture you'll also need fetchcode and storecode.
Implementing an Umbilical Forth this minimalist way may lead to
slow interaction, especially when Flash programming has to be
taken into account.
You should also note that starting from the three-instruction
idea and getting to a seamless interactive Forth requires a
great deal of patience and attention to detail - it's not a
quick job.
There were about three Rochester Forth conference papers on
the topic in the 1980s.
>Brad is of course very interested in how to implement such a kernel in
>an optimized way. But again "make it work, then make it fast." comes to
>mind. In my estimation one needs two sets of items to execute forth words:
>
>1. The forth virtual machine including the standard registers and
>stacks.
>
>2. Three critical words: ENTER, EXIT, and NEXT.
>
>In short create enough Forth to execute the inner interpreter and you
>can get going.
Don't make implementation assumptions at this stage! For
example, Chuck Moore might well say that an eight-deep return
stack is always enough. Providing that you know about it,
that's enough for serious work. It's even better if you
can force a satisfying crash on overflow.
It's no coincidence that both Forth Inc and MPE focus on
native code compilers, and MPE at least is much more likely
to use a byte-code token system when space is an issue rather
than an ITC or DTC system.
Stephen
--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
none Byron Jeff
Elizabeth D Rather <erath...@forth.com> wrote:
>none Byron Jeff wrote:
>> In article <137rks1...@news.supernews.com>,
>> Elizabeth D Rather <erath...@forth.com> wrote:
>>> none Byron Jeff wrote:
>...
>>> outlined, but it appears as though it's really handcuffing you in terms
>>> of designing a workable development cycle.
>>
>> I don't think so. Execution effiency isn't my primary goal, and that's
>> what will be handcuffed. I was working with a bytecode interpreter that
>> is probably averaging 40 instructions per bytecode on average. At the
>> time I started working on it years ago, my bytecode memory was a
>> bitbanged serial EEPROM. Execution effiency isn't a worry. It's the item
>> I can most afford to give up initially.
>
>You are handcuffed in the sense that you would like to be able to
>download small amounts of code into ram and execute it. You don't seem
>to have enough ram to do this, not to mention not enough stack space,
>etc. Hence, a civilized development environment will be very much more
>difficult to arrange than on other platforms.
I see that I missed explaining something. Code isn't going into RAM. The
code is burned into the flash of the PIC. There's 4-8K of flash on a
typical 16F part now. And the only parts I'll even consider are ones
that can programmatically program their own flash.
There's more than enough space for code. Even if I tokenize, I wouldn't
put those tokens in RAM. RAM is strictly for stacks and variables.
The flash is the reason that I'm harping on the incremental development
model. It's slow to program, much faster to read. The flash is the
Harvard architecture part of the PIC. The Harvard architecture also
dictates that native code cannot be run from RAM anyway.
>>> I don't see how the issue of incremental compilation and downloading is
>>> dependent on the execution model.
>>> Regardless of what your compiled
>>> stuff looks like (machine instructions, addresses, or tokens) you still
>>> have to be able to download little bits of it and execute it, right?
>>
>> The little bits is the key. PicForth and Mary are organized as a one
>> shot compilation environment. You compile the entire system and download
>> it at one go. I want pretty much the opposite. The current tools I have
>> available to me are not organized that way.
>
>Right. But the problem is that the tools are designed for the
>limitations of the platform. A less limited platform can more easily
>support the kind of tools you're seeking.
We've established that. And I would agree with you if I were really
trying to run tokens from out of RAM. But it's must less constrained
than I think you thought the PIC is.
>>> Unless (I'm really pretty ignorant of PIC architecture) you have a
>>> Harvard architecture and it's code space you have no access to.
>>
>> It's a Harvard architecture that's difficult to access and slow to
>> update. So transferring little bits at a time is a highly desireable
>> trait.
>
>Yes. But you need somewhere to transfer them to that's a little more
>accessible than that. PIC doesn't seem to support ram development,
>which is the best way to do incremental testing.
This is the reason I'm wanting to use the host as a remote execution
environment. The model you see is:
1. Compile words on the host
2. Transfer compiled code words to RAM.
3. Test/Debug code on target recompiling and reloading as necessary.
4. When finished transfer code to more permanent memory on target.
A perfectly workable model.
The model I envision:
1. Prototype/test/debug words on host using tether to manipulate I/O
2. Compile words for target substituting local I/O access for remote ones.
3. Transfer words to target flash.
4. Retest word on target.
The model that I don't want:
1. Compile all words on host
2. Transfer all compiled words to flash on target
3. Debug/test on target.
4. Rinse and repeat.
I can work that model perfectly in assembly, Jal, C, NPCI, Basic, or
any number of traditional laguages/development models.
>>> In that
>>> case, either addresses or tokens could work (which is why we used tokens
>>> on the AVR 8515).
>>
>> Tokens is the winner in my case too. Still a bit concerned about if I'm
>> constrained in my token size (or if it really matters).
>
>Not really.
From what I was reading in Brad's article it seems to be possible to
implement the token interpreter so that variable length tokens are
doable. It's a model that I had already implemented for my bytecode
interpreter, so I have no problem with it. So for now that's what I'll
plan to do reserving some of the 8 bit token space for extended word
tokens. It'll make my life on the host a bit more difficult because I'll
need to essentially do some hamming encoding of words to make sure that
the most frequently used words are in the smallest token space. But
again that's an optimization issue, not an implementation one.
>
>>> ...
>>>> But in my self-chosen constrained target environment this approach fails
>>>> on several levels given the goals I hope to achieve:
>>>>
>>>> 1. The pic's hardware stack is limited. Subroutine calls are simply not
>>>> an option because the stack overflows after only 8 levels of calls.
>>> Well, it's somewhat limiting, but shouldn't be fatal. Most Forth apps
>>> aren't really nested very deeply. We've run some pretty hairy apps on
>>> 8051's with on-chip stacks of limited size.
>> That's encouraging. I just worry about reliability because if you
>> overflow the hardware stack, your application is guaranteed to crash
>> eventually.
>> And that stack is completely unmapped in memory. It's probably the thing
>> about the part that drives me the most crazy because you can't implement
>> any effective context switching without access to that stack.
>Among the many reasons we avoid using PICs.
And probably one of the reasons that no one has taken on the challenge
of implementing anything other than a batch forth compiler for it.
I think that you may have convinced me to take a stab at an STC
implementation. It does eliminate the need for the address interpreter
and will speed up execution of the most critical part of the threading
mechanism. It'll save me the code space of a token table. It would be an
absolute no brainer of a decision for an 18F part (32 level addressible
stack). I figure that if I don't implement recursion it should be fine.
And if not, I know that I have tokens as a backup.
>> Exactly how limited were those 8051 stacks?
>
>64 bytes (32 cells) as I recall. Could have been 48 bytes. It's been a
>while.
Now we are talking about the hardware subroutine stack right? Just
making sure.
>>>> 2. Taking this route commits you to compiling your entire application
>>>> because once you do away with the inner interpreter, then everything on
>>>> the target must be compiled.
>>> No, it doesn't. Changing to direct code compilation didn't affect our
>>> development cycle at all. Unless you're saying this based on another
>>> PIC-specific obstacle we haven't heard about yet.
>>
>> Not sure. I think I'm reaching the boundaries of my understanding. If
>> point #1 above is taken off the table, then I think I can see it because
>> implementing optimized STC eliminates the interpreter yet facilitates
>> incremental additions to the codebase on the target. But if that
>> hardware stack is out of bounds, I'm lost as to how you could implement
>> ITC, DTC, or TTC without elements of the address interpreter.
>The execution model doesn't affect whether you can or cannot do
>incremental compilation. What determines that is whether you have a
>place to put the downloaded definitions to test them. You make it sound
>like the address interpreter is a big deal. It isn't.
From my reading (primarily Brad's articles) it's the core concept of
interpreted Forth. Without it you're back to the traditional compilation
model of inline expansion of native code. I already have a compiler like
that languishing on my hard disk. Not real interested in writing
another.
>
>...
>> But it gets back to the question of how much kernel do you have to
>> download to get started? At the very least in bootstrapping the kernel a
>> distributed model would have to be in play.
>
>You start with a couple dozen bytes and use that to load the rest. We
>find a few K to be capable of providing a useful set of primitives.
>
>...
>>
>> I guess my question is what is the structure of an optimized compiled
>> code word then? What I cannot visualize is the linkages between the code
>> fragments.
>
>It's code in code space. Small primitives or optimized code sequences
>are expanded in place. Larger words are called. Linkage is normal
>call/return.
So it's STC with inline expansion of smaller fragments. Got it.
>> I think that structurally I can easily see how to compile a definition
>> into a collection of addresses or tokens. However compiling native code
>> is a different animal.
>Simpler, really.
If it's straight STC you're right it's simpler. Still a bit worried
about the PIC hardware call/return stack. If it overflows, your
application goes into the weeds.
>
>>> ...
>>>> Microcontrollers also have mechanisms for dealing with time critical
>>>> stuff. Another reason I love using PICs is the wide variety of hardware
>>>> periperals they come packaged. UARTS, multiple timers, PWM, ADC, and the
>>>> like are really set/autopilot types of tools. Interrupts can be used to
>>>> buffer really time sensitive stuff.
>>> So do most modern microcontrollers. Take another look at some of the
>>> alternatives. There are some pretty nice parts out there.
>>
>> I'm aware. Remember I got here because I was looking at the propeller.
>> I'm already having to get up to speed with a new language and a new
>> tool. Componding it by starting from scratch with a new architecture is
>> too much to tackle.
>
>It's a lot easier to come up to speed with a new language if you have
>thoroughly tested, well-documented tools at hand. Trying to develop
>tools for a language that's new to you is a much worse challenge.
It's new only to a point. If I were starting at ground zero I might
agree. But having implemented a stack based bytecode interpreter for the
target already, I'm way ahead of the game. I fundamentally already have
the XTL executive in place. It's simply a matter of linking in higher
level forth words into it. I feel I'm much less likely to get lost
starting bottom up with the foundation I know than restarting top down
to an unfamiliar base. In my current spot, forth is the only unknown,
and these discussions with you and others are clearing that up fairly
quickly.
>
>> Plus I feel if I can pull this off in my constrained little box, that
>> moving the port to a roomier chip (like the propeller, which of course
>> due to Cliff I don't need to do) should be no problem.
>
>Isn't that like saying, if I can learn to ride a unicycle, a tricycle
>will be easy?
No. It's saying I already know how to ride a unicycle, so the trike is
trivial.
>>>> If all else fails after developing the application, simply run it all
>>>> through an optimizing compiler removing the inner interpreter altogether
>>>> along with other connecting tissue beween words.
>>> That shouldn't have to be an extra step. An optimizing compiler isn't a
>>> post-processor, it's an *alternative* to another kind of compiler (such
>>> as ITC).
>>
>> An incremental optimizing Forth compiler for the PIC 16F platform
>> doesn't exist AFAICT. It needs to be built. My experience with language
>> tool building and with pics tells me that the optimizing compiler is the
>> much tougher road to travel to get to a incremental development target.
>
>True, but even developing an interactive, incremental ITC or token-based
>development system for this beast will be extremely difficult.
The token based executive is already done. See my other post for the
description of the bytecode interpreter with my comments.
Bruce threw together about a dozen primitives based on a simple one
register model along with about 7 primitives in a post. Forth doesn't
need much of a base to get rolling.
>> A non incremental optimizing compiler does exist. But I doesn't suit my
>> development needs.
>
>It was developed by a very clever guy. I'm sure if he could have built
>an incremental, interactive compiler he would have.
People develop tools based on their perceptions. Actually I read on his
site that he had started down the path I'm on now and decided that it
would be quicker to implement a cross compiler. He also had a particular
target in mine (model train stuff).
All of my perceptions are colored from my pervious experiences. For me
projects go better when I can noodle with stuff. Most time I spiral from
the middle of a specification in both directions. It works for me.
So I want to build the tools that fits that development style. In
addition I want tools that fir into my current toolset. Meeting forth in
the last couple of weeks has again changed my perception of what that
tool needs to be.
>> Put the two together and the answer that pops out is to implement a non
>> optimized token based compiler. I'm in my wheelhouse there because I
>> already have a token based, stack implemented 16F kernel that's already
>> tested and can be quick adapted to the task.
>>
>> While I do have fun building tools, they do have the purpose of building
>> other stuff. I prefer building the simplest foolproof tools I can build
>> then using them to bootstrap up.
>>
>> Implementing an address interpreter with NEXT, ENTER, and EXIT "words"
>> will cost me an afternoon and about 25-30 lines of assembly. Then I'll
>> have a tool that I can use to put forth on my target.
>>
>> I'm not worried about slow. I'm worring about getting done and having
>> the right result when I get done.
>
>Well, it sounds as though you have months of fun ahead of you.
Months? What makes you think that. There are at least a 1/2 dozen forths
out here that are predicated on operating on top of a simplified kernel.
The kernel already exists. All that's required is a simple mapping
between the primitives required by a particular forth and the existing
kernel.
Is there some magic incantation that I'm missing here?
>...
>>>>>> Only one piece of the puzzle is missing at this point. Elizabeth
>>>>>> discusses in her post above that the XTL transfers the stack between the
>>>>>> host and the target. In short it implements a form of distributed
>>>>>> execution where you muster the stacks for RPC. In doing so one can run a
>>>>>> application with a set of words distributed between the host and the
>>>>>> target.
>>> Well, it only models the data stack on the host. The return stack stays
>>> on the target. And target words only execute on the target. There's no
>>> attempt to simulate execution of target words on the host.
>>
>> Ah. I see. So that means that your XTL had to be significantly developed
>> before you could start using it. The appeal of Frank's paper was that
>> essentially once you implemented his three instructions kernel, that you
>> could immediately start developing applications with it without needing
>> to flesh out an entire kernel just to get started. This leads back to
>> the point I made in my initial post that a good (albeit slow) small set
>> of primitives would be good to implement. And the 48 that I've seen for
>> MAF doesn't qualify as a small set.
>
>The target side of the XTL is very small and simple, probably not
>significantly different from Frank's concept. The host is rather more
>complex. And the concepts have been developed over about 20 years.
>There's a lot of advantages to be found in "standing on the shoulders of
>giants".
OK I'll take you on with this. Here are my requirements:
1. I need an environment that runs natively on my Linux box.
2. I need the specification of the XTL and the host wire interface.
3. I need complete documentation of the host environment.
Can you provide that? How much will it cost me?
>> I see the distributed model sort of as a breakpoint. The host already
>> has everything (primitives, core words, core extensions) already
>> implemented. Why not use it as a remote process server in addition to
>> the text interpreter, wordlist coordinator, and the target's cross
>> compiler?
>
>Yes.
You said no previously. Something to the effect that all words ran on
the target and that the stack was not transferred between the host and
the target.
So which is it?
>
>...
>>> A classic Forth
>>> has a text interpreter, which processes text from a user or disk and
>>> generates ("compiles") executable definitions (which might be actual
>>> code, strings of addresses of words to be executed, tokens, or some
>>> other internal form). That which used to be called an "inner"
>>> interpreter, more accurately "address" interpreter, processes the
>>> strings of addresses in the "compiled" form of a definition if that's
>>> the model being used. It's usually only 1-3 machine instructions per
>>> address, although on some processors it's more.
>>
>> Right. The point of running Forth on the host is to get a complete
>> environment without having the burden the target with it. This is the
>> tethered model. But no one seems to be addressing the possibility of
>> distributed computing between the host and the target. The host is
>> simply a respository for a set of services (text interpreter, cross
>> compiler, wordset dictionaries) without helping the target run any
>> actual forth code. The way I see it since the host is a full forth
>> environment, it can emulate a full forth environment for the target.
>
>Yes, the host provides all those services. What it doesn't do well is
>execute target code. Therefore, we make no attempt to execute target
>code on the host, but transparently exercise it on the target.
I understand. We don't need the host to execute target code. The host is
running forth, the target is running forth. I do realize that they may
be implemented differently (different cell sizes and the like). But
forth is forth is forth. So if there's a word on the host that the
target can use, why is it necessary to compile that word for the target,
transfer the word to the target and execute the word on the target? An
RPC model where the host can execute a word (or set of words) is also a viable
model during development.
[snippage]
>> BTW I realized that I'm still trying to figure out how in the heck forth
>> compiles a number into a definition. What is the xt for a number?
>
>It's the address for (or call to) a word that pushes the actual value on
>the stack, followed by the value:
>
> ... [xt of LIT] [value] ...
So the number is inlined into the code.
>
>There may be different versions for 8, 16, or 32-bit literals. It has
>to advance the interpreter pointer or PC beyond the value, in addition
>to pushing the value on the stack.
That's problematic for an STC that doesn't have access to the hardware
return stack. It'll have to be inlined.
BAJ
Coos Haak
> In article <137sb0r...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>>none Byron Jeff wrote:
<snip>
>>> It's a Harvard architecture that's difficult to access and slow to
>>> update. So transferring little bits at a time is a highly desireable
>>> trait.
>>
>>Yes. But you need somewhere to transfer them to that's a little more
>>accessible than that. PIC doesn't seem to support ram development,
>>which is the best way to do incremental testing.
>
> This is the reason I'm wanting to use the host as a remote execution
> environment. The model you see is:
>
> 1. Compile words on the host
> 2. Transfer compiled code words to RAM.
> 3. Test/Debug code on target recompiling and reloading as necessary.
> 4. When finished transfer code to more permanent memory on target.
>
> A perfectly workable model.
I doubt it, what about inline data (like literals, branches etc.)
You have to have two different token interpreters: one for executing code
out of data memory (that would be RAM I assume) and one for code in code
memory. One of them would have to have an extra indirection. As they say
around here: two believers in one bed, the devil sleeps between them.
Look at the 8051s, here it is possible to overlay code and data regions, so
the CPU does not know it's executing from RAM.
<snip>
>>> BTW I realized that I'm still trying to figure out how in the heck forth
>>> compiles a number into a definition. What is the xt for a number?
>>
>>It's the address for (or call to) a word that pushes the actual value on
>>the stack, followed by the value:
>>
>> ... [xt of LIT] [value] ...
>
> So the number is inlined into the code.
Code or data space?
>
>>
>>There may be different versions for 8, 16, or 32-bit literals. It has
>>to advance the interpreter pointer or PC beyond the value, in addition
>>to pushing the value on the stack.
>
> That's problematic for an STC that doesn't have access to the hardware
> return stack. It'll have to be inlined.
>
Many Forths synthesize the second stack, be it the return or data one, no
problem here.
--
Coos
CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html
none Byron Jeff
Stephen Pelc <steph...@INVALID.mpeforth.com> wrote:
>On Sat, 23 Jun 2007 12:21:29 -0500, byron@upstairs.(none) (Byron Jeff)
>wrote:
>
>Elizabeth has explained enough of the mechanics that I won't repeat
>here explanations. By the way, I'm a Stephen not a Steven.
I'll keep that in mind. BTW thanks so much for making your excellent
book available online. It's helped me get up to speed.
>The problem with PICs for Forth is that the early ones are based
>on a 1970s architecture for a minicomputer I/O processor. The
>benefits of PICs are in the price and the peripherals. An
>undoubted benefit for hobbyists is that you can still get them
>in DIP packages.
That along with the fact that they are virtually indestructable makes
them an excellent platform for hobby use.
>For commercial vendors of Forth, the price point of the
>standard tools does not make PICs a very attractive target,
>especially as high-end PICs are more expensive than low-end
>ARMs. The PIC18 was the first Forth-friendly PIC, but these
>days has little to offer unless you are already a PIC user.
Exactly. Being an existing user and tool developer is probably the only
reason I'm still contemplating implementing forth on that platform.
>Within MPE, the question of supplying a PIC system is raised
>every year or so, and we always end up saying "not unless
>a client sponsors it".
And with other tools available that may be slow in coming. However
having freely available hobby forths for the platform out there may
generate enough interest someday to make a commercial version a
possibility.
>>So the question is "Given an embedded micro, what is the minimum forth
>>environment required to execute Forth words?".
>
>It has been done many times using: fetch, store, execute. For a
>Harvard architecture you'll also need fetchcode and storecode.
Now is there an underlaying specification of how to implement the rest
of the core words using those primitives? BTW I did take a quick peek at
the smartcard tokenized VM. At first glance it seemed heavyweight.
>Implementing an Umbilical Forth this minimalist way may lead to
>slow interaction, especially when Flash programming has to be
>taken into account.
Slow is OK for starters. That's why Frank's implementation shows a
glimmer of merit. Couple it with Bruce's example simplified model
outlined in his other post and it doesn't seem to be too terribly
complicated to get started with a slow forth.
I realize that the real challenge is making a fast, compact memory
footprint forth. But my methodology is to iterate towards that as a
goal. Initially even if it's two words per second, it's good if it
works.
>You should also note that starting from the three-instruction
>idea and getting to a seamless interactive Forth requires a
>great deal of patience and attention to detail - it's not a
>quick job.
But as I've pointed out I actually have a written, tested, and working
stack based executive kernel from my NPCI bytecode interpreter. It took
a while for me to write and test it initially. But it's fairly complete
and can quickly be pressed into service. All I needed was a hook to get
an umbilical forth going to that executive. Frank's idea provides that
bridge.
>There were about three Rochester Forth conference papers on
>the topic in the 1980s.
I'll go back and take a look for them. Maybe forth.org has them online.
>>Brad is of course very interested in how to implement such a kernel in
>>an optimized way. But again "make it work, then make it fast." comes to
>>mind. In my estimation one needs two sets of items to execute forth words:
>>
>>1. The forth virtual machine including the standard registers and
>>stacks.
>>
>>2. Three critical words: ENTER, EXIT, and NEXT.
>>
>>In short create enough Forth to execute the inner interpreter and you
>>can get going.
>
>Don't make implementation assumptions at this stage! For
>example, Chuck Moore might well say that an eight-deep return
>stack is always enough. Providing that you know about it,
>that's enough for serious work. It's even better if you
>can force a satisfying crash on overflow.
The last part is what I worry about. The pic hardware stack is a
circular buffer. There isn't any error condition when the stack is
overwritten. It does it sliently. Eventually the program will go into
the weeds and there will be no indication as to when that happened.
But I'll take your's and Elizabeth's expert advise on being able to
function with an 8 deep stack. The STC (with a bit of inlining) is in
fact the most simple and the most efficient code generation technique
off the bat.
>
>It's no coincidence that both Forth Inc and MPE focus on
>native code compilers, and MPE at least is much more likely
>to use a byte-code token system when space is an issue rather
>than an ITC or DTC system.
It only took understanding the fetch process of a program memory word to
eliminate each of these two as a possbility. I know tokens are doable
because NPCI operates on tokens. But STC would be the most efficient
implementation.
Thanks for the comments,
BAJ
none Byron Jeff
Coos Haak <chf...@hccnet.nl> wrote:
>Op Sun, 24 Jun 2007 10:12:40 -0500 schreef none:
>
>> In article <137sb0r...@news.supernews.com>,
>> Elizabeth D Rather <erath...@forth.com> wrote:
>>>none Byron Jeff wrote:
><snip>
>>>> It's a Harvard architecture that's difficult to access and slow to
>>>> update. So transferring little bits at a time is a highly desireable
>>>> trait.
>>>
>>>Yes. But you need somewhere to transfer them to that's a little more
>>>accessible than that. PIC doesn't seem to support ram development,
>>>which is the best way to do incremental testing.
>>
>> This is the reason I'm wanting to use the host as a remote execution
>> environment. The model you see is:
>>
>> 1. Compile words on the host
>> 2. Transfer compiled code words to RAM.
>> 3. Test/Debug code on target recompiling and reloading as necessary.
>> 4. When finished transfer code to more permanent memory on target.
>>
>> A perfectly workable model.
>
>I doubt it, what about inline data (like literals, branches etc.)
Coos, I was describing Elizabeth's model of development, not the one I'm
proposing.
>You have to have two different token interpreters: one for executing code
>out of data memory (that would be RAM I assume) and one for code in code
>memory. One of them would have to have an extra indirection. As they say
>around here: two believers in one bed, the devil sleeps between them.
>Look at the 8051s, here it is possible to overlay code and data regions, so
>the CPU does not know it's executing from RAM.
So then exactly how do you untether the target from the host? Again I
was describing the model that Elizabeth described to me in her previous post.
><snip>
BTW you snipped my proposed model without comment. Code only transfers
once to the target and it goes directly to flash. No need for tracking
two different token streams.
>>>> BTW I realized that I'm still trying to figure out how in the heck forth
>>>> compiles a number into a definition. What is the xt for a number?
>>>
>>>It's the address for (or call to) a word that pushes the actual value on
>>>the stack, followed by the value:
>>>
>>> ... [xt of LIT] [value] ...
>>
>> So the number is inlined into the code.
>Code or data space?
Code. PICs are incapable of execute code directly from RAM.
I believe that pushing a number would have to be inlined in an STC
implementation because the hardware return stack is inaccessible. So you
can't read it in order to get the address of the value to be pushed, nor
can you modify the stack in order to return back to the code stream
after the value upon return. finally you can't leave the return address
on the stack and jump back to another location because the stack is
circular and limited and you'll overflow it. In short in an STC
implementation you can't make a call to a word to push the actual value.
Hence you'll have to inline it and put the actual code to push the value
in the code stream.
Of course all of this will go into flash where it can be executed
directly.
>>>There may be different versions for 8, 16, or 32-bit literals. It has
>>>to advance the interpreter pointer or PC beyond the value, in addition
>>>to pushing the value on the stack.
>>
>> That's problematic for an STC that doesn't have access to the hardware
>> return stack. It'll have to be inlined.
>>
>Many Forths synthesize the second stack, be it the return or data one, no
>problem here.
The data stack will be synthesized, as will a nominal return stack. But
implementing with STC means using the PIC's hardware return stack. It's
unmapped in memory. It's invisible. Can't get to it.
Coos, the PIC 16F family is a weird bird. The only reason I'm still in
it is because of years of investment. Funnily enough forth may finally
free me from it. But to do it, I still feel the need to implement a
workable forth for me on that platform.
BAJ
Bruce McFarling
> I see that I missed explaining something. Code isn't going into RAM. The
> code is burned into the flash of the PIC. There's 4-8K of flash on a
> typical 16F part now. And the only parts I'll even consider are ones
> that can programmatically program their own flash.
Remember that a typical code definition will be two to seven words
long ... sometimes longer, but even then if there is a length
constraint, you can factor it down to a set of shorter words. So if
stored in Flash will be 14bit words, so that stored in Ram will be
2bytes (with the top two bits free), a 64 byte "loading dock", or
"BAY" in RAM to hold the data to be flashed is certainly workable.
And you got the RAM for that, given a 32 cell data stack and a 16 cell
return stack, plus the hardware stack as an auxillary stack for
primitives if its quicker to use than a software stack.
So the umbilical kernel could be:
EXECUTE: execution token
ACCUMULATE: data (length counter of BAY incremented by one)
STORE: length ... error return if length does not match length counter
in BAY, reset length to 0.
RETRIEVE: length ... report contents of BAY to master, length at the
beginning ... error return if comment length is longer than length
counter in BAY.
... with 14 bit execution tokens, four functions can be encoded in the
two high bits of a two-byte word.
and some of first primitives that you store:
* take an accumulated string of data in the BAY and puts in the RAM
location as addressed in the first token in the string (partial high
byte disregarded)
* use the address and length of a pair of tokens in the BAY to load
data from the RAM to the bay (length to BAY length counter)
* read off the data in the BAY, starting with the BAY length counter.
* copy a string from program memory based on address and length of a
pair of tokens in the BAY
That gives you the ability to load a primitive at a time, test them
individual by copy contents to the stack locations and then reading
the stack locations back, then colon definitions and do the same type
of testing.
This way, you don't need the full kernel to debug the individual
primitives. Of course, your EXECUTE does need to be in sync with your
threading model, so whether there is a token interpreter, or bit
threaded / token interpreter, or a DTC explicit ENTER/EXIT/NEXT, that
machinery will be in the Kernal and used by EXECUTE.
none Byron Jeff
Bruce McFarling <agi...@netscape.net> wrote:
>On Jun 24, 11:12 am, byron@upstairs.(none) (Byron Jeff) wrote:
>> I see that I missed explaining something. Code isn't going into RAM. The
>> code is burned into the flash of the PIC. There's 4-8K of flash on a
>> typical 16F part now. And the only parts I'll even consider are ones
>> that can programmatically program their own flash.
>
>Remember that a typical code definition will be two to seven words
>long ... sometimes longer, but even then if there is a length
>constraint, you can factor it down to a set of shorter words. So if
>stored in Flash will be 14bit words, so that stored in Ram will be
>2bytes (with the top two bits free), a 64 byte "loading dock", or
>"BAY" in RAM to hold the data to be flashed is certainly workable.
That's just about exactly the right size. Most of the more modern of the
16F PICs require writing to flash in 8 word chunks and erasing in 32
word rows. To truthfully a 16-20 byte dock would be more workable.
>And you got the RAM for that, given a 32 cell data stack and a 16 cell
>return stack, plus the hardware stack as an auxillary stack for
>primitives if its quicker to use than a software stack.
I'm going to take a first crack at a STC with some inlining. So the
hardware stack will actually be the word return stack.
>So the umbilical kernel could be:
>EXECUTE: execution token
Good.
>ACCUMULATE: data (length counter of BAY incremented by one)
This too. While it could be factored into send individual bytes, sending
a 16 byte packet along with 2 byte address is a better model. I wouldn't
bother with a length byte, the packet has to be exactly 16 bytes because
that's the smallest chunk that can be written to flash anyway.
>STORE: length ... error return if length does not match length counter
>in BAY, reset length to 0.
Now that's interesting. I think that ACCUMULATE and STORE can be
combined. The only problem I see is that since erasure requires a 32
word row, there's have to be a way to copy a entire row to RAM, update
it in ram, erase the row in flash, then rewrite the updated row to
flash.
The cheap and simple to get started is to initally write words only in a
32 word alignment, erasing the entire row before writing. It'll cause
too much internal fragmentation in practice, but would be fine for a
proof of concept.
>RETRIEVE: length ... report contents of BAY to master, length at the
>beginning ... error return if comment length is longer than length
>counter in BAY.
Maybe RETRIEVE can alleviate the problem above. The host can simply read
the entire row from the executive, update it as needed, ask the target
to erase the row, then fire back the row as 4 consecutive STORES.
Also to read the flash there'd be no need to buffer the data, simply
read it from the flash and fire it directly down the wire. So that would
facilitate keeping the BAY down in size.
>... with 14 bit execution tokens, four functions can be encoded in the
>two high bits of a two-byte word.
>
>and some of first primitives that you store:
>* take an accumulated string of data in the BAY and puts in the RAM
>location as addressed in the first token in the string (partial high
>byte disregarded)
replace RAM with flash and I'm with you here.
>* use the address and length of a pair of tokens in the BAY to load
>data from the RAM to the bay (length to BAY length counter)
Same here.
>* read off the data in the BAY, starting with the BAY length counter.
Do you mean transfer the BAY to the host?
>* copy a string from program memory based on address and length of a
>pair of tokens in the BAY
>
>That gives you the ability to load a primitive at a time, test them
>individual by copy contents to the stack locations and then reading
>the stack locations back, then colon definitions and do the same type
>of testing.
>
>This way, you don't need the full kernel to debug the individual
>primitives. Of course, your EXECUTE does need to be in sync with your
>threading model, so whether there is a token interpreter, or bit
>threaded / token interpreter, or a DTC explicit ENTER/EXIT/NEXT, that
>machinery will be in the Kernal and used by EXECUTE.
Understood. Shouldn't be a problem with STC. fundamentally word
definitions will compile to a series of calls.
Thanks for the insight.
BAJ
Elizabeth D Rather
> In article <137sb0r...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>> You are handcuffed in the sense that you would like to be able to
>> download small amounts of code into ram and execute it. You don't seem
>> to have enough ram to do this, not to mention not enough stack space,
>> etc. Hence, a civilized development environment will be very much more
>> difficult to arrange than on other platforms.
>
> I see that I missed explaining something. Code isn't going into RAM. The
> code is burned into the flash of the PIC. There's 4-8K of flash on a
> typical 16F part now. And the only parts I'll even consider are ones
> that can programmatically program their own flash.
>
> There's more than enough space for code. Even if I tokenize, I wouldn't
> put those tokens in RAM. RAM is strictly for stacks and variables.
>
> The flash is the reason that I'm harping on the incremental development
> model. It's slow to program, much faster to read. The flash is the
> Harvard architecture part of the PIC. The Harvard architecture also
> dictates that native code cannot be run from RAM anyway.
Thanks, it's a lot clearer now. But now a few more questions.
Is all your flash code space, or can you set it to be part code, part
data? Because if you're doing tokens, wouldn't your token tables go in
data space, with your token interpreter and primitives in code space?
In any case, even 8K is by no means generous for the code you'll need
plus token tables.
...
>
> The model I envision:
>
> 1. Prototype/test/debug words on host using tether to manipulate I/O
> 2. Compile words for target substituting local I/O access for remote ones.
> 3. Transfer words to target flash.
> 4. Retest word on target.
The difficulty I see with this model is that your host is *very*
different from your target. You can't, for example, test any of your
PIC code (of which there will be some, for your primitives at least,
regardless of what model you use) on the host. And unless you make a
token interpreter on the host as well, there's no way to test that logic.
> The model that I don't want:
>
> 1. Compile all words on host
> 2. Transfer all compiled words to flash on target
> 3. Debug/test on target.
> 4. Rinse and repeat.
>
> I can work that model perfectly in assembly, Jal, C, NPCI, Basic, or
> any number of traditional laguages/development models.
I certainly sympathize with not wishing that model.
...
>>> Tokens is the winner in my case too. Still a bit concerned about if I'm
>>> constrained in my token size (or if it really matters).
>> Not really.
>
> From what I was reading in Brad's article it seems to be possible to
> implement the token interpreter so that variable length tokens are
> doable. It's a model that I had already implemented for my bytecode
> interpreter, so I have no problem with it. So for now that's what I'll
> plan to do reserving some of the 8 bit token space for extended word
> tokens. It'll make my life on the host a bit more difficult because I'll
> need to essentially do some hamming encoding of words to make sure that
> the most frequently used words are in the smallest token space. But
> again that's an optimization issue, not an implementation one.
The usual approach is to have 1-byte tokens for your most common words,
maybe 250 of them, and then use the few remaining tokens (e.g. FE) to
signal that there will be a 2nd token used against a secondary table.
So, unless you're planning really ambitious apps (which you're not) you
never need more than 2-byte tokens.
...
>
> I think that you may have convinced me to take a stab at an STC
> implementation. It does eliminate the need for the address interpreter
> and will speed up execution of the most critical part of the threading
> mechanism. It'll save me the code space of a token table. It would be an
> absolute no brainer of a decision for an 18F part (32 level addressible
> stack). I figure that if I don't implement recursion it should be fine.
Well, that certainly precludes testing your definitions on the host in
more than a cursory fashion.
> And if not, I know that I have tokens as a backup.
>
>>> Exactly how limited were those 8051 stacks?
>> 64 bytes (32 cells) as I recall. Could have been 48 bytes. It's been a
>> while.
>
> Now we are talking about the hardware subroutine stack right? Just
> making sure.
Yes. On the most limited 8051 parts we don't use the Forth return stack
for return addresses, but the subroutine stack.
...
>
>> The execution model doesn't affect whether you can or cannot do
>> incremental compilation. What determines that is whether you have a
>> place to put the downloaded definitions to test them. You make it sound
>> like the address interpreter is a big deal. It isn't.
>
> From my reading (primarily Brad's articles) it's the core concept of
> interpreted Forth. Without it you're back to the traditional compilation
> model of inline expansion of native code. I already have a compiler like
> that languishing on my hard disk. Not real interested in writing
> another.
The address interpreter is a core concept of indirect-threaded code
Forth. Other models work differently, and all may support incremental
compilation.
>> ...
>>> I guess my question is what is the structure of an optimized compiled
>>> code word then? What I cannot visualize is the linkages between the code
>>> fragments.
>> It's code in code space. Small primitives or optimized code sequences
>> are expanded in place. Larger words are called. Linkage is normal
>> call/return.
>
> So it's STC with inline expansion of smaller fragments. Got it.
>
>>> I think that structurally I can easily see how to compile a definition
>>> into a collection of addresses or tokens. However compiling native code
>>> is a different animal.
>
>> Simpler, really.
>
> If it's straight STC you're right it's simpler. Still a bit worried
> about the PIC hardware call/return stack. If it overflows, your
> application goes into the weeds.
If small primitives are expanded in line, there's less requirement for
calls.
...
>>> Plus I feel if I can pull this off in my constrained little box, that
>>> moving the port to a roomier chip (like the propeller, which of course
>>> due to Cliff I don't need to do) should be no problem.
>> Isn't that like saying, if I can learn to ride a unicycle, a tricycle
>> will be easy?
>
> No. It's saying I already know how to ride a unicycle, so the trike is
> trivial.
But you *don't* know how to ride a unicycle, in the sense that you don't
have your incremental, interactive development system on a PIC, and I
suspect you're glossing over the difficulties of building one. A
token-based executive, which you say you have, is the tip of the iceberg.
...
...
>> The target side of the XTL is very small and simple, probably not
>> significantly different from Frank's concept. The host is rather more
>> complex. And the concepts have been developed over about 20 years.
>> There's a lot of advantages to be found in "standing on the shoulders of
>> giants".
>
> OK I'll take you on with this. Here are my requirements:
>
> 1. I need an environment that runs natively on my Linux box.
> 2. I need the specification of the XTL and the host wire interface.
> 3. I need complete documentation of the host environment.
>
> Can you provide that? How much will it cost me?
More than is appropriate for a hobby/educational project (mostly the
cost of the Linux port). But the documentation is at least free if you
get a SwiftX evaluation package.
>>> I see the distributed model sort of as a breakpoint. The host already
>>> has everything (primitives, core words, core extensions) already
>>> implemented. Why not use it as a remote process server in addition to
>>> the text interpreter, wordlist coordinator, and the target's cross
>>> compiler?
>> Yes.
>
> You said no previously. Something to the effect that all words ran on
> the target and that the stack was not transferred between the host and
> the target.
>
> So which is it?
Yes it's reasonable to use the host to provide the text interpreter,
hold and manage dictionary heads, manage wordlists, and provide a cross
compiler. And I don't think I said the stack is *not* transferred
between the host and target, because it very definitely *is*.
No it's not reasonable to think that you can test your app definitions
usefully by running them on the host. The host is more different than
you realize.
>> ...
> I understand. We don't need the host to execute target code. The host is
> running forth, the target is running forth. I do realize that they may
> be implemented differently (different cell sizes and the like). But
> forth is forth is forth. So if there's a word on the host that the
> target can use, why is it necessary to compile that word for the target,
> transfer the word to the target and execute the word on the target?
Because you need to test whether the target version works. It's
different, regardless of the implementation strategy you use. To take a
very basic example, you can assume that gForth's DUP works. How will
you know PIC's DUP works without testing it on the PIC? If you're
compiling STC for the PIC, how can you know you're doing it correctly
without testing the generated code on the PIC? If you're using tokens,
how do you know the token interpreter is working and tokens are being
generated correctly without testing that on the PIC? Yes, to a limited
extent you can test some high-level logic on the host, but that's the
least of what you need to worry about.
Gerry
> >relatively simple cross compiler that is relatively easy to convert to
> >another platform. It is a little confusing at first because many low
> >level words exist as machine level words and are only binary codes.
>
> That means that it isn't necessarily minimal. Maybe I should stop
> complaining because I have my own bytecode interpreter that has about 35
> instructions. I factored the binary ops by writing a common routine that
> pops the stack into a temp register (binopsetup). So most of those
> instructions were little more than doing a binopsetup followed by the
> actual task of the instruction, then writing back the result to the new
> TOS. That accounted for about half the group
> (+,-,=,!=,>,<,>=,<=,&,|,^,&&,||,<<,>>)
>
> An interesting question is how many of these could you actually factor?
> All 6 comparison operators are buildable from a subtract, as is addition
> if you throw in ^. AND an OR are negative complements of one another so
> you really only need NAND to build both (and ^ too IIRC). && and || are
> buildable from comparison and logical ops. Left shift can be done via
> recursive addition. Not sure about right shift.
>
> So the kernel for the 15 above tentatively is: -, NAND, >>
>
> You then throw in the equivalents for ! and @, unary ops, pushing
> numbers on the stack, conditional and unconditional goto, and you're
> pushing 20. Since I was targeting a traditional language I didn't throw
> in but a couple of stack manipulation operators. I guess implementing
> DROP, SWAP, DUP, OVER would be good. Any others?
>
I'm a bit surprised that nobody has mentioned Buzzard to you (unless I
missed it). See
http://www.ioccc.org/1992/buzzard.2.design
which develops a small Forth system from 15 primitives.
Gerry
none Byron Jeff
Bruce McFarling <agi...@netscape.net> wrote:
>On Jun 24, 8:15 pm, byron@upstairs.(none) (Byron Jeff) wrote:
>> >ACCUMULATE: data (length counter of BAY incremented by one)
>> This too. While it could be factored into send individual bytes,
>>sending a 16 byte packet along with 2 byte address is a better model. I
>>wouldn't bother with a length byte, the packet has to be exactly 16
>>bytes because that's the smallest chunk that can be written to flash
>>anyway.
>My idea was that the BAY can also be used to read and write to the
>stacks, to set up the initial stack and read out the resulting stack,
>when testing whether primitives work. So a string of 14-bit values,
>with the top two bits telling the function (I'd use 00 for
>ACCUMULATE). If the destination out of the BAY is data memory, the
>upper 6 bits of the 14 bit value is just ignored.
OK that makes sense. You'd then need to have a length marker because the
size of the stacks are variable.
>
>> >STORE: length ... error return if length does not match length
>> > counter in BAY, reset length to 0.
>
>> Now that's interesting. I think that ACCUMULATE and STORE can be
>> combined.
>
>Again, the double purpose of using ACCUMULATE to set up the BAY for
>transfers to and from RAM or for Flash is why the STORE to flash is
>seperate ... once you have execute code in flash, download data, and
>store data to flash, you can build the balance of the umbilical
>development system from there.
Got it. The BAY is a generic transfer point.
>> The only problem I see is that since erasure requires a 32
>> word row, there's have to be a way to copy a entire row to RAM,
>> update it in ram, erase the row in flash, then rewrite the
>> updated row to flash.
>I don't see why. Just make sure that MARKER's are aligned on word row
>boundaries, so that when the MARKER is executed it cleans out all the
>Flash from that point on.
But would that not limit words to 32 word boundaries in the flash? Fine for
initial testing by wasteful in practice.
>>>RETRIEVE: length ... report contents of BAY to master, length at the
>>>beginning ... error return if comment length is longer than length
>>>counter in BAY.
>
>> Also to read the flash there'd be no need to buffer the data, simply
>> read it from the flash and fire it directly down the wire. So that
>> would facilitate keeping the BAY down in size.
>
>Neither reading the RAM to check the stack effects of primitives as
>they are loaded ... so BAY can be dedicated for input from the host.
So the host is limited to sending packets of a constrained size, but the
target can send packets of the requested size directly down the wire.
That asymmetry makes sense.
>> >* take an accumulated string of data in the BAY and puts in the RAM
>> >location as addressed in the first token in the string (partial high
>> >byte disregarded)
>
>> replace RAM with flash and I'm with you here.
>
>That's STORE. This primarily is to load the stacks with parameters for
>testing.
There's really no need to have a separate command to differentiate RAM
and Flash. This is a discussion I already had with my students. The
address byte is going to be 16 bits. No pic address space comes close to
that. So simply subdivide the address space so that low address go to
RAM and high addresses go to flash. Then all you need is a single STORE
to determine to where the BAY is transferred.
You can even have an address space for the data EEPROM so that you can
transfer data to and from it.
>> >* use the address and length of a pair of tokens in the BAY to load
>> >data from the RAM to the bay (length to BAY length counter)
>
>> Same here.
>As noted above, this need not go through the BAY, it can be output by
>the PIC directly
Right. The host will have a virtually unlimited BAY. In addition since
the host specifies how many cells it wants, it too can limit the size of
the packets coming back from the target if it wants.
>> >* read off the data in the BAY, starting with the BAY length counter.
>
>> Do you mean transfer the BAY to the host?
>Yes, but given direct read flash and read RAM operations, this becomes
>redundant.
No need for a double transfer. Got it.
>> Understood. Shouldn't be a problem with STC. fundamentally word
>> definitions will compile to a series of calls.
>
>... with the last word in the definition being a jump instead.
Woozy with confusion... I thought the last word would be a return. STC
uses the hardware PC as the "IP". So when you finish a word you retrieve
where you came from from the hardware return stack. That's a return.
Only exception would be code that's inlined into the instruction stream.
Elizabeth has already help me identify that literals would have to be
inlined for example.
>Precisely ... and as long as you know you are working with an eight
>deep R stack, that's certainly workable. That also simplifies EXECUTE
>a lot.
Have to think about EXECUTE. I'm less woozy from for initial suggestion
about a jump at the end now. The problem is that we're not working in
RAM and we don't have access to the hardware return stack. BTW this will
be a 3 stack machine because we still need a R stack in addition to the
hardware return stack and the data stack. So let's differentiate by
having a R stack and a HR stack, OK? The HR stack is 8 deep and
invisible. Also we're working out of flash. So we can neither easily set
up a stack entry to return from to start executing, nor can we easily
setup a computed call (that would be very useful here). This is
complicated by the fact that for ordinary words we utilize standard
call/return semantics.
What's going to be needed is the concept of a special top level word
that both gets the ball rolling and knows to return to the executive
when it finishes. A MAIN word as it were. The easiest way to implement
this is to have a word that jumps to the executive and put it as the
last word of the MAIN definition (EXIT?). Another possibility (which would
slow down overall execution) is that we put that EXIT word at the end of
every definition and make it conditional based on some flag. If the flag
is set, we jump to the executive, unset, we return normally. I still
have to think how we could use such a construct because the flag needs
to be set just before you execute the EXIT token. Can't think of how to
make this transparent.
Great ideas. Thanks for the input.
BAJ
none Byron Jeff
Elizabeth D Rather <erath...@forth.com> wrote:
>none Byron Jeff wrote:
>> In article <137sb0r...@news.supernews.com>,
>> Elizabeth D Rather <erath...@forth.com> wrote:
>> ...
>> I see that I missed explaining something. Code isn't going into RAM. The
>> code is burned into the flash of the PIC. There's 4-8K of flash on a
>> typical 16F part now. And the only parts I'll even consider are ones
>> that can programmatically program their own flash.
>>
>> There's more than enough space for code. Even if I tokenize, I wouldn't
>> put those tokens in RAM. RAM is strictly for stacks and variables.
>>
>> The flash is the reason that I'm harping on the incremental development
>> model. It's slow to program, much faster to read. The flash is the
>> Harvard architecture part of the PIC. The Harvard architecture also
>> dictates that native code cannot be run from RAM anyway.
>
>Thanks, it's a lot clearer now. But now a few more questions.
>
>Is all your flash code space, or can you set it to be part code, part
>data?
The flash is readable in a couple of different ways. So data can be
embedded into it. The slow way is to programmatically read the flash. It
has the advantage of giving you access to all 14 bits of each location
facilitating 14 bit tokens. The disadvantage is that reading in this
manner is really slow, like 15-20 instructions cycles slow. The
advantage is that you could theoretically have 32k tokens, though the
token lookup table would in fact fill the flash for the part.
The faster way is using the PIC RETLW instruction which performs a
return and sticks a 8 bit value into the W register. Coupled with the
PIC computed GOTO capability it's possible to get this 8 bit value in
about 4-5 instruction cycles. But the cost is that you lose nearly half
of the usable bits from the word. BTW this cost is just the cost to
fetch the token. You still need to do the indirect table lookup and call
to the routine in either instance.
>Because if you're doing tokens, wouldn't your token tables go in
>data space, with your token interpreter and primitives in code space?
Nope. The tokens go into code space as outlined above. The second method
is exactly how I built the NPCI get_token routine. It can fetch a 8 bit
token from anywhere in the flash.
>In any case, even 8K is by no means generous for the code you'll need
>plus token tables.
It'll be a tight squeeze no doubt. IIRC the current NPCI bytecode
interpreter clocks in at a bit under 2K. But it currently implements a
complete 8 bit and 16 bit set of stack routines along with code for a
frame pointer (C like language). So with some trimming I probably can
get it back down towards 1K.
Also with some encouragement, I've decided that my first crack at this
is going to be a STC. So I'm keeping tokens in my back pocket for now.
STC facilitates not having to limit the numer of words and the execution
speed is going to be probably 8-20 times faster than either of the token
methods.
>
>...
>>
>> The model I envision:
>>
>> 1. Prototype/test/debug words on host using tether to manipulate I/O
>> 2. Compile words for target substituting local I/O access for remote ones.
>> 3. Transfer words to target flash.
>> 4. Retest word on target.
>
>The difficulty I see with this model is that your host is *very*
>different from your target. You can't, for example, test any of your
>PIC code (of which there will be some, for your primitives at least,
>regardless of what model you use) on the host. And unless you make a
>token interpreter on the host as well, there's no way to test that logic.
Tokens are off the table for now, though a token interpreter in forth
wouldn't be that difficult to pull off. I in fact started writing a
token emulator in C precisely so that I could do testing on the host
before dealing with the target.
But what I don't think that you see is that I'm interested in executing
forth, not PIC code, on the host. I'm not concerned about primitives
since the executive is already written.
But you do raise an interesting point of how could forth on the host be
used to assist in the testing of primitives on the target. It brings
several assets to the table:
1. It can assemble and download native PIC code. At 35 instructions I
would guess that a PIC assembler would not take too terribly long to
construct in forth.
2. Via the tether you can program the PIC (a big win by the way. I'm a
bootloader lover) execute code (Frank's third instruction), view memory,
and exercise I/O if necessary.
3. It can be used as a cross compiler for forth words. The existing
PicForth compiler generates native PIC code.
If I didn't have the executive substatively written, I'd look to writing
it in forth and getting a small kernel going so that I could tether the
target.
What to do if PICforth or Mary didn't exist at all is a discussion for
another day.
>
>> The model that I don't want:
>>
>> 1. Compile all words on host
>> 2. Transfer all compiled words to flash on target
>> 3. Debug/test on target.
>> 4. Rinse and repeat.
>>
>> I can work that model perfectly in assembly, Jal, C, NPCI, Basic, or
>> any number of traditional laguages/development models.
>
>I certainly sympathize with not wishing that model.
That's the only model that I've had in the past for doing development.
Even worse the traditional development model inserts an actual
programmer in the middle of it. Drives me nuts. All of my successful
projects have been done working off a tether with a bootloader.
>
>...
>>>> Tokens is the winner in my case too. Still a bit concerned about if I'm
>>>> constrained in my token size (or if it really matters).
>>> Not really.
>>
>> From what I was reading in Brad's article it seems to be possible to
>> implement the token interpreter so that variable length tokens are
>> doable. It's a model that I had already implemented for my bytecode
>> interpreter, so I have no problem with it. So for now that's what I'll
>> plan to do reserving some of the 8 bit token space for extended word
>> tokens. It'll make my life on the host a bit more difficult because I'll
>> need to essentially do some hamming encoding of words to make sure that
>> the most frequently used words are in the smallest token space. But
>> again that's an optimization issue, not an implementation one.
>
>The usual approach is to have 1-byte tokens for your most common words,
>maybe 250 of them, and then use the few remaining tokens (e.g. FE) to
>signal that there will be a 2nd token used against a secondary table.
>So, unless you're planning really ambitious apps (which you're not) you
>never need more than 2-byte tokens.
That's what I figured. Somehow in all of this reading I got the
impression that tokens/code had to fit some fixed sized mold. Totally
incorrect of course.
>
>...
>
>>
>> I think that you may have convinced me to take a stab at an STC
>> implementation. It does eliminate the need for the address interpreter
>> and will speed up execution of the most critical part of the threading
>> mechanism. It'll save me the code space of a token table. It would be an
>> absolute no brainer of a decision for an 18F part (32 level addressible
>> stack). I figure that if I don't implement recursion it should be fine.
>
>Well, that certainly precludes testing your definitions on the host in
>more than a cursory fashion.
Cursory is all I wanted. Being able to test logic and flow without
having to compile and download each time is a big win. It's somewhat a
part of my development style as it is. I generally use a PIC emulator to
test code logic before dealing with real hardware. Helps to get gross
misconceptions out of the way before firing up real hardware.
In fac the emulator is how I'll test all of this development before
firing up any real hardware (which my interns are working on right now).
>
>> And if not, I know that I have tokens as a backup.
>>
>>>> Exactly how limited were those 8051 stacks?
>>> 64 bytes (32 cells) as I recall. Could have been 48 bytes. It's been a
>>> while.
>>
>> Now we are talking about the hardware subroutine stack right? Just
>> making sure.
>
>Yes. On the most limited 8051 parts we don't use the Forth return stack
>for return addresses, but the subroutine stack.
Got it. That's where I am.
>
>...
>>
>>> The execution model doesn't affect whether you can or cannot do
>>> incremental compilation. What determines that is whether you have a
>>> place to put the downloaded definitions to test them. You make it sound
>>> like the address interpreter is a big deal. It isn't.
>>
>> From my reading (primarily Brad's articles) it's the core concept of
>> interpreted Forth. Without it you're back to the traditional compilation
>> model of inline expansion of native code. I already have a compiler like
>> that languishing on my hard disk. Not real interested in writing
>> another.
>
>The address interpreter is a core concept of indirect-threaded code
>Forth. Other models work differently, and all may support incremental
>compilation.
I see that now. They are in fact completely orthogonal. It just so
happened that the two Pic Forth cross compilers happened to be
constructed in a non incremental fashion.
>
>>> ...
>>>> I guess my question is what is the structure of an optimized compiled
>>>> code word then? What I cannot visualize is the linkages between the code
>>>> fragments.
>>> It's code in code space. Small primitives or optimized code sequences
>>> are expanded in place. Larger words are called. Linkage is normal
>>> call/return.
>>
>> So it's STC with inline expansion of smaller fragments. Got it.
>>
>>>> I think that structurally I can easily see how to compile a definition
>>>> into a collection of addresses or tokens. However compiling native code
>>>> is a different animal.
>>
>>> Simpler, really.
>>
>> If it's straight STC you're right it's simpler. Still a bit worried
>> about the PIC hardware call/return stack. If it overflows, your
>> application goes into the weeds.
>
>If small primitives are expanded in line, there's less requirement for
>calls.
I'll keep that in mind.
I have to go. I'll tackle the rest when I get a chance....
BAJ
none Byron Jeff
Gerry <ge...@jackson9000.fsnet.co.uk> wrote:
>
>I'm a bit surprised that nobody has mentioned Buzzard to you (unless I
>missed it). See
>
>http://www.ioccc.org/1992/buzzard.2.design
>
>which develops a small Forth system from 15 primitives.
This is the first that I've seen it. Other than having a multiply, FIRST
is virtually perfect for implementing a tiny kernel supporting a Forth
subset.
Will examine further.
Thanks so much
BAJ
marc....@telenet.be
FlashForth (http://www.kolumbus.fi/oh2aun/) from Mikael Nordman looks
a bit like what you describe, though it's for the PIC 18Fxxx(x)
series...
Hope this helps...
Marc
Bruce McFarling
> >I don't see why. Just make sure that MARKER's are aligned on word row
> >boundaries, so that when the MARKER is executed it cleans out all the
> >Flash from that point on.
> But would that not limit words to 32 word boundaries in the
> flash? Fine for initial testing by wasteful in practice.
Sorry, speaking in jargon. MARKER is a defining word, and the defined
word, when executed removes, itself and every definition defined after
it from the dicationary. So ONLY the MARKER words would have to be row-
aligned.
> >> Understood. Shouldn't be a problem with STC. fundamentally word
> >> definitions will compile to a series of calls.
>
> >... with the last word in the definition being a jump instead.
> Woozy with confusion... I thought the last word would be a return. STC
> uses the hardware PC as the "IP". So when you finish a word you retrieve
> where you came from from the hardware return stack. That's a return.
The last word in the sequence in compiled code does not have to be a
call with a return at the end:
* if the last word called is a primitive, it will end with a return
* if it is not, then *it* will end with a jump to a word, and
somewhere down there, the last thing done is a primitive, and all
primitives end in a return.
Of course, if it makes the colon compiler simpler, that can be left
for later refininement.
none Byron Jeff
I'm back for another round after thinking about it a couple of days. I'm
coming to the stark realization that I'm really a Forth novice. My lack
of Forth experience seems to have two spheres of influence:
1. A lack of understanding of how Forth words "do stuff". To really
leverage the power of Forth, one really needs to have both a grasp of
the available wordsets and an understanding of how to adapt the words in
those wordsets to one's needs.
2. The environment lends itself to developing with a Forthlike thought
process. I've grasped a bit of it by reading "Thinking Forth". But with
a firm grounding in languages like C, it takes some experience to adapt
thought processes to the new model.
In short I have a pretty good idea what I want to do, but I'm not sure
either if I'm thinking about the right way, or even if I am thinking
about it the right way, how to pull it off.
So I come hat in hand asking for some help in both thinking about the
problem and how to go about developing it. It doesn't have to be a
detailed analysis I don't think, just some guidance of where to look for
some answers.
Problem Definition: I want to develop an incremental word cross compiler
for the PIC. I hope this has been a "been there, done that." type
exercise for some of you. The idea is to develop and test the word on
the host, then cross compile the word into its PIC equivalent for
transport to the target. So for example say I develop a simple word on
the host:
: add2 2 + ;
I test it out to see that it works:
5 add2 . 7 ok
Now I'd like to put the word permanently on the pic. Say for the sake of
argument I generate a string with the name of the word then call the
word picompile which generates the equivalent word for the pic:
S" add2" picompile ok
Say for the sake of argument the code for the pic equivalent is
generated in the word add2.pic. In addition the address for the word on
the pic is stored in the dictionary under add2.picaddr. I could then
somehow download the word to the PIC:
S" add2" picload ok
where picload uses the add2.picaddr and add.pic code to transfer the
code to the pic.
End of problem definition.
Now I've probably screwed up things 6 ways from Sunday. But I'm willing
to learn some proper ways of expressing the ideas that I've come up
with.
Some of my thought process has been colored by Brad Rodriguez's article
on developing a cross assembler in Forth:
http://www.zetetics.com/bj/papers/tcjassem.txt
what he wrote made a lot of sense. But I need to adapt some ideas.
Questions:
1. Instead of taking input from the normal input buffer, I want to
fundamentally parse a dictionary definition. The way I see it is
something like running "see <word>" taking the output and parsing it.
What I'm out of the loop with is accessing dictionary definitions
directly and parsing the words in those definitions.
2. I know that create puts new words in the dictionary and allot will
allocate space. I'm unsure how to access that allotted space once I
created it or exactly how to get a pointer to the definition's alloted
area. Brad used C, to extend and write into the dictionary space for
example.
3. I used strings above due to ignorance. How could one simplify the
examples to
add2 picompile
add2 picload
for example?
4. I know that Forth parses and does dictionary lookups. What kinds of
words should I be looking for to do these activities? Can parsing be
done from an arbitrary buffer?
I'm sure I have more, but I think this is enough to motivate some
discussion on the subject. Am I thinking about the problem in terms that
are even remotely Forthlike? Or would another approach make more sense.
Thanks for any thoughts on the matter,
BAJ
Elizabeth D Rather
> Hi folks,
>
> I'm back for another round after thinking about it a couple of days. I'm
> coming to the stark realization that I'm really a Forth novice. My lack
> of Forth experience seems to have two spheres of influence:
Yes, this is why I recommended you get a free SwiftX cross-compiler and
experiment with it for a while. Write some Forth apps and make them
work on some processor SwiftX supports (since the boards are cheap and
you aren't developing any new hardware it should be non-traumatic).
> Questions:
>
> 1. Instead of taking input from the normal input buffer, I want to
> fundamentally parse a dictionary definition. The way I see it is
> something like running "see <word>" taking the output and parsing it.
> What I'm out of the loop with is accessing dictionary definitions
> directly and parsing the words in those definitions.
>
> 2. I know that create puts new words in the dictionary and allot will
> allocate space. I'm unsure how to access that allotted space once I
> created it or exactly how to get a pointer to the definition's alloted
> area. Brad used C, to extend and write into the dictionary space for
> example.
>
> 3. I used strings above due to ignorance. How could one simplify the
> examples to
>
> add2 picompile
> add2 picload
>
> for example?
The normal way of things is that you load on your host a cross-compiler
that looks just like the resident compiler, but generates PIC code. So,
no such thing as picompile, just process the source in the same way but
you get pic code, ideally automatically downloaded.
> 4. I know that Forth parses and does dictionary lookups. What kinds of
> words should I be looking for to do these activities? Can parsing be
> done from an arbitrary buffer?
Ideally it should replicate the resident system, but generate PIC code
and download it transparently. Use the same parsing words, same logic,
just feed to a different version of :, CREATE, VARIABLE, etc.
> I'm sure I have more, but I think this is enough to motivate some
> discussion on the subject. Am I thinking about the problem in terms that
> are even remotely Forthlike? Or would another approach make more sense.
>
> Thanks for any thoughts on the matter,
You haven't read the XC standard docs yet, have you?
Cheers,
Elizabeth
Frank Buss
> Say for the sake of argument the code for the pic equivalent is
> generated in the word add2.pic. In addition the address for the word on
> the pic is stored in the dictionary under add2.picaddr. I could then
> somehow download the word to the PIC:
>
> S" add2" picload ok
While developing your Forth system, maybe it is easier to emulate the PIC
on the PC. Then you don't need to worry about how to flash the code and if
you write the emulator in Forth, you'll learn a lot about Forth, too.
Bonus: A PIC disassembler, to see what your compiler produced (but this
could be included in the code generator, like I've done with my FPGA Forth
implementation).
--
Frank Buss, f...@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
none Byron Jeff
Frank Buss <f...@frank-buss.de> wrote:
>none wrote:
>
>> Say for the sake of argument the code for the pic equivalent is
>> generated in the word add2.pic. In addition the address for the word on
>> the pic is stored in the dictionary under add2.picaddr. I could then
>> somehow download the word to the PIC:
>>
>> S" add2" picload ok
>
>While developing your Forth system, maybe it is easier to emulate the PIC
>on the PC.
That's certainly my plan.
>Then you don't need to worry about how to flash the code and if
>you write the emulator in Forth, you'll learn a lot about Forth, too.
No need. I can use the excellent gpsim simulator located here:
The PIC side of things is a no brainer for me.
Also I've learned not to dive knee deep into peripheral projects. gpsim
is nearly 10 years old and is still a work in progress. I'm not about to
tackle another emulator.
>Bonus: A PIC disassembler, to see what your compiler produced (but this
>could be included in the code generator, like I've done with my FPGA Forth
>implementation).
Again included in gpsim, so it would be adding to the department of
redundancy department (of redundancy).
BAJ
none Byron Jeff
>> Hi folks,
>>
>> I'm back for another round after thinking about it a couple of days. I'm
>> coming to the stark realization that I'm really a Forth novice. My lack
>> of Forth experience seems to have two spheres of influence:
>...
>
>Yes, this is why I recommended you get a free SwiftX cross-compiler and
>experiment with it for a while. Write some Forth apps and make them
>work on some processor SwiftX supports (since the boards are cheap and
>you aren't developing any new hardware it should be non-traumatic).
Elizabeth,
All the above suggestion does is divert focus away from my real problem.
You outline 4 things that I don't really need:
1. A Forth product I don't plan to use.
2. Embedded systems hardware I don't plan to buy.
3. Applications that I don't need to write.
4. A development model that doesn't fit the model I want to use.
As we have discussed elsewhere in this thread, I'm only a novice to the
Forth language. For programming in general, embedded systems development,
and the like I know my way around the block.
>> Questions:
>>
>> 1. Instead of taking input from the normal input buffer, I want to
>> fundamentally parse a dictionary definition. The way I see it is
>> something like running "see <word>" taking the output and parsing it.
>> What I'm out of the loop with is accessing dictionary definitions
>> directly and parsing the words in those definitions.
>>
>> 2. I know that create puts new words in the dictionary and allot will
>> allocate space. I'm unsure how to access that allotted space once I
>> created it or exactly how to get a pointer to the definition's alloted
>> area. Brad used C, to extend and write into the dictionary space for
>> example.
>>
>> 3. I used strings above due to ignorance. How could one simplify the
>> examples to
>>
>> add2 picompile
>> add2 picload
>>
>> for example?
>
>The normal way of things is that you load on your host a cross-compiler
>that looks just like the resident compiler, but generates PIC code. So,
>no such thing as picompile, just process the source in the same way but
>you get pic code, ideally automatically downloaded.
I got that from my readings. But it doesn't really address the model of
testing words on the host, then transferring words to the target. The
reason is that most forth embedded systems targets are RAM based and
therefore can be loaded at wire speed. However, in flash based
environments, the flash write speed often dominates the transfer time.
In addition flash does have some (and often severe) write cycle
limitations. So a "test many, write once" model has some merit.
I believe I have good reasons to explore an alternate model of testing
words on the host before transferring to the target. It fits my existing
embedded systems development model. Frank Buss in the other reply
validated it by pointing out that testing in a emulator would be a good
idea.
I'm trying to find ways to adapt forth to the environment that I have
instead of adapting the environment to fit forth's expectations. I'm
just trying to do some quality control in the design stages to ensure
that adaptation is done in a forth-like way.
>> 4. I know that Forth parses and does dictionary lookups. What kinds of
>> words should I be looking for to do these activities? Can parsing be
>> done from an arbitrary buffer?
>
>Ideally it should replicate the resident system, but generate PIC code
>and download it transparently.
See above.
>Use the same parsing words, same logic,
>just feed to a different version of :, CREATE, VARIABLE, etc.
OK that makes sense to a point. In the XC stuff that I've read (along
with Stephen's chapter on the subject in the Programming Forth book)
this is done via words that defines the environment (wordset?) you're
currently operating on. Switches are done via words such as HOST,
TARGET, INTERPRETER, and the like.
What does not seem to be addressed is my specific point of wanting words
that exist in one environment (HOST) to transfer and compile that word
on the other side of the fence. In other words I'm looking to avoid the
following sequence:
HOST
: add2 2 + ;
5 add2 . ( do a bit of testing ) 7 ok
( it works so let's get it to the target )
TARGET
: add2 2 + ( why do I need to retype it? ) ;
I already have the word I want in the host wordset. I don't want to have
to repeat typing it (and all of its error filled possibilities) in the
target wordset.
Transparency is useful once you have the underlaying system working.
However, it's not your friend when you're trying to understand how to
develop the infrastructure you're wanting to implement. That's why I
motivated my example with an explicit word for the activity.
The transparency I'd really be interested in moving forward is the
automatic transferrance of yet to be compiled words. For example say we
created and tested the following definitions on the host:
: add2 2 + ;
: add4 add2 add2 ;
5 add4 . 9 ok
Being happy with the result, let's get the word into target space:
add4 picompile ok
Now add4 refers to a word (add2) that's currently not compiled in the target
space. It would be nice to have a transparent compilation of that
additional word into the target space too.
>> I'm sure I have more, but I think this is enough to motivate some
>> discussion on the subject. Am I thinking about the problem in terms that
>> are even remotely Forthlike? Or would another approach make more sense.
>>
>> Thanks for any thoughts on the matter,
>
>You haven't read the XC standard docs yet, have you?
I read some. Doesn't seem to fit my particular requirements. If I really
wanted the follow the standard, I should follow your suggestions at the
top of this post.
To reiterate my questions:
1. Words for accessing dictionary definitions like SEE does?
2. Parsing from somewhere other than the input buffer. Or alternatively
how to transfer text into the input buffer without typing it?
3. words to address allotted memory in the dictionary?
Thanks,
BAJ
Jonah Thomas
> Problem Definition: I want to develop an incremental word cross
> compiler for the PIC. I hope this has been a "been there, done that."
> type exercise for some of you.
Yes, it is.
> The idea is to develop and test the word on
> the host, then cross compile the word into its PIC equivalent for
> transport to the target. So for example say I develop a simple word on
> the host:
>
> : add2 2 + ;
>
> I test it out to see that it works:
>
> 5 add2 . 7 ok
>
> Now I'd like to put the word permanently on the pic.
Say you have a Forth interpreter/compiler that can both run Forth, and
it also has a PIC assembler and a way to send code to the PIC.
Ideally you'd set it up so you can just switch back and forth. Do FORTH
and it compiles things into the host and interprets things on the host.
5 add2 . will first put 5 on the host stack, add2 turns it to 7, and .
displays it. Do PIC and now 5 sends a 5 to the PIC's stack, add2 will
look up the address of the add2 command that's compiled on the PIC and
tells the PIC to execute that, and . tells the PIC to send the TOS back
to the host to be displayed.
Doesn't that seem like a sort of ideal for ease-of-use? You might want a
few more commands available. but that gives you most of it. Switch to
PIC and do : FOO and you add a name FOO to the host dictionary, and
start compiling a routine on the PIC, and get the address of the start
of the routine to put into FOO on the host. Then each PIC routine you
access gets added to the definition on the PIC until you get to ; which
compiles an exit on the PIC .
So you need a few special commands. : IF THEN ; etc. Things that do
special compilation tricks etc, that you can and should some of the
calculation on the host and then add the result to the PIC .
If you look at the cross-compiler standard, it will give you suggested
solutions to all of the problems. Or at least, all the ones that aren't
so well-known in Forth that they forgot to mention them. You don't have
to do it their way, but it's probably easier to see what they're doing
than figure it all out from first principles.
> Questions:
>
> 1. Instead of taking input from the normal input buffer, I want to
> fundamentally parse a dictionary definition. The way I see it is
> something like running "see <word>" taking the output and parsing it.
> What I'm out of the loop with is accessing dictionary definitions
> directly and parsing the words in those definitions.
If you have a definition in a string, with most Forths you can give the
address and length of the string and then do EVALUATE and it will parse
the string. If the definition is in a series of strings with line breaks
you can EVALUATE them one at a time. Since it's a single-pass evaluation
you don't have to worry about going back and picking up things from an
old buffer. Single-pass evaluation avoids so many problems I strongly
suggest you avoid multi-pass evaluation for your special PIC compiler.
> 2. I know that create puts new words in the dictionary and allot will
> allocate space. I'm unsure how to access that allotted space once I
> created it or exactly how to get a pointer to the definition's alloted
> area. Brad used C, to extend and write into the dictionary space for
> example.
CREATE FOO 500 CELLS ALLOT
FOO . will give you the address of the beginning of the buffer. It's
like FOO is a variable, it gives you the address, but instead of one
cell you can allot as much space as you want. Was that what you wanted,
or were you asking how to implement that?
> 3. I used strings above due to ignorance. How could one simplify the
> examples to
>
> add2 picompile
> add2 picload
You want to compile the definition on the host first, test it, and then
compile it onto the PIC . Or maybe you'd like to compile PIC code onto
the host and test it with a simulator and then download it to the PIC.
I'd suggest the first way first -- if there's a problem with your logic,
fix that before you get involved with whether your simulator or your
assembler have a problem. So FORTH to compile and test the definition in
Forth. Then SIM to compile and test the definition on the simulator.
Then PIC to compile and test the definition on the PIC .
Now, you want to use the name of the definition to point to the
definition so you can do things with it? Or point to the code so you
don't hvae to recompile the PIC code for the PIC after you've already
done it once for the simulator? You can do that. Or you could make a
file named add2.txt and take it from there, or really whatever you like.
> 4. I know that Forth parses and does dictionary lookups. What kinds of
> words should I be looking for to do these activities? Can parsing be
> done from an arbitrary buffer?
Yes, EVALUATE parses from an arbitrary buffer.
You can look up one word with FIND or, differently, the more modern word
SEARCH-WORDLIST .
You can parse one word from the current input buffer with WORD or the
more modern word PARSE-NAME (or PARSE-WORD ).
But look, you're asking very basic Forth questions. If you're going to
use Forth you might as well figure out how. If you asked a whole lot of
questions that were in the FAQ at some point people would start pointing
you to the FAQ . And it really makes sense to point you to a Forth
manual.
Maybe you could start some simple project in Forth to get clearer about
the language. Lots of people write a simple Forth compiler to learn the
language, which unfortunately tends to give them more experience at
writing Forth compilers than at using Forth. (Unless they write most of
the Forth compiler in Forth.) Maybe it would be useful for you to take
some project you've already done and translate it into Forth. Not a
literal translation, figure out how to use Forth tools to do it simpler.
Maybe start by looking for things that get done more than once that you
can factor out. Short definitions that become tools. Then later you
might see ways to throw away whole sections and replace them with
something simpler. And the point of this isn't just to improve your
product (though making it simpler may remove some subtle bugs). It's to
learn all the Forth tricks that you haven't thought to ask us about yet,
that will make your main project much easier to create.
Jonah Thomas
> All the above suggestion does is divert focus away from my real
> problem. You outline 4 things that I don't really need:
>
> 1. A Forth product I don't plan to use.
> 2. Embedded systems hardware I don't plan to buy.
> 3. Applications that I don't need to write.
> 4. A development model that doesn't fit the model I want to use.
>
> As we have discussed elsewhere in this thread, I'm only a novice to
> the Forth language. For programming in general, embedded systems
> development, and the like I know my way around the block.
I can sympathise. Sometimes it seems like there's always something else
to do first before you can do the thing you want.
But if you want to use Forth, you need to learn how to use Forth. That
doesn't need to take a long time, but it's a necessary step. So get some
Forth compiler and practice with it. Elizabeth's product isn't the only
free one that works, but her manual is very good. If you use a different
one, choose one with a good Forth manual. Write some sort of
applications to learn Forth. You learn more by doing than by watching or
pretending to do. You don't have to use somebody else's cross-compiler
model but the more of it you can use the less you have to re-invent for
yourself. That's a plus.
> >The normal way of things is that you load on your host a
> >cross-compiler that looks just like the resident compiler, but
> >generates PIC code. So, no such thing as picompile, just process the
> >source in the same way but you get pic code, ideally automatically
> >downloaded.
>
> I got that from my readings. But it doesn't really address the model
> of testing words on the host, then transferring words to the target.
> The reason is that most forth embedded systems targets are RAM based
> and therefore can be loaded at wire speed. However, in flash based
> environments, the flash write speed often dominates the transfer time.
> In addition flash does have some (and often severe) write cycle
> limitations. So a "test many, write once" model has some merit.
I see! So you might want to write and test a whole block of code with
your simulator, and then when you can fill a whole block of your flash
then transfer them all at once.
On the host you'd still want to keep track of the addresses to call on
the PIC so you could test them easily. It looks like the concession
you're making to reality here is that you want to flash blocks at a time
and do your final testing for whole blocks of words at once, in the hope
that you'll only have to do it again a few times.
> >Use the same parsing words, same logic,
> >just feed to a different version of :, CREATE, VARIABLE, etc.
>
> OK that makes sense to a point. In the XC stuff that I've read (along
> with Stephen's chapter on the subject in the Programming Forth book)
> this is done via words that defines the environment (wordset?) you're
> currently operating on. Switches are done via words such as HOST,
> TARGET, INTERPRETER, and the like.
>
> What does not seem to be addressed is my specific point of wanting
> words that exist in one environment (HOST) to transfer and compile
> that word on the other side of the fence. In other words I'm looking
> to avoid the following sequence:
>
> HOST
> : add2 2 + ;
> 5 add2 . ( do a bit of testing ) 7 ok
> ( it works so let's get it to the target )
>
> TARGET
> : add2 2 + ( why do I need to retype it? ) ;
>
> I already have the word I want in the host wordset. I don't want to
> have to repeat typing it (and all of its error filled possibilities)
> in the target wordset.
One way is to put the word in a file, and load the file. And then you
might as well keep your tests in a file too. Maybe keep a history file
and edit away the parts you don't want. When I first started I used to
do whatever tests I happened to think of at the keyboard. That worked
very well. But your test suite is part of the documentation, and if you
someday wind up doing it again or something similar, why should you have
to re-invent everything from scratch? And if you later find an error
your tests didn't uncover, you might notice why you didn't think to
check it the first time around, and that will improve your testing
skills. So if you're going to put all that stuff in files anyway, why
not just use them again? And then there isn't much improvement to
compile once and copy the code to the PIC, versus recompile. You save
processor time, which is completely irrelevant. On the other hand, if
you use absolute addressing or absolute jumps, you *need* to recompile.
> : add2 2 + ;
> : add4 add2 add2 ;
> 5 add4 . 9 ok
>
> Being happy with the result, let's get the word into target space:
>
> add4 picompile ok
You can do that. The easy way, you do
' add2 picompile
' add4 picompile
so you don't have to look up each command it calls to make sure they're
already compiled.
You want your code to use relative jumps and relative calls, right? So
in the best case you can just take your block of code and pass it to the
PIC and flash it. That shouldn't be too hard.
> Now add4 refers to a word (add2) that's currently not compiled in the
> target space. It would be nice to have a transparent compilation of
> that additional word into the target space too.
You can do that too but it's a little harder. You want to compile each
of the words it calls, recursively. You want to recurse your way down to
a word that doesn't call anything new and compile that before you start
compiling anything else. I think you'll probably have to give up
single-pass compilation to do that, but maybe not. It's potentially a
lot of complication compared to just checking for yourself. If it was me
doing it, I'd think about how much code I was going to write that needed
it, and I'd probably put off doing that step until I noticed I was
making errors without it. If you put your time into whatever looks like
the limiting factor now, it's likely to give you faster progress.
> 1. Words for accessing dictionary definitions like SEE does?
Like SEE ? Ah. You get an xt, or anyway the address of a call, and you
want to find the name. The newer Forth standards don't give you any way
to do that since some Forth systems can't do it easily. The Forth-83
standard let you do that, I think. There was a word like >NAME . I don't
remember whether you had to do >BODY >NAME or some other combination.
You might check the documentation of your particular Forth, or look at
the code with DUMP or something like that to see how it's actually laid
out.
> 2. Parsing from somewhere other than the input buffer. Or
> alternatively how to transfer text into the input buffer without
> typing it?
EVALUATE . or LOAD for up to 1024 characters.
> 3. words to address allotted memory in the dictionary?
HERE 256 cells allot CONSTANT MY-DATA
ok
MY-DATA U.
ok 21063284
Give it the name to get the base address. Add whatever offset you want
from there to get later addresses.
CREATE MY-DATA2 32 CELLS ALLOT
MY-DATA2 acts just like MY-DATA except the code gets compiled first.
When the one thing that's most getting in your way is that you don't
know Forth, it looks like learning Forth would be the limiting factor
that deserves some effort.
Brad Eckert
>
> Problem Definition: I want to develop an incremental word cross compiler
> for the PIC. I hope this has been a "been there, done that." type
> exercise for some of you. The idea is to develop and test the word on
> the host, then cross compile the word into its PIC equivalent for
> transport to the target. So for example say I develop a simple word on
> the host:
>
> : add2 2 + ;
Here are a couple of options:
1. The compiler creates a word ADD2 on the host that pushes 2 onto the
target's stack and tells the target to execute the "+" token. That
tests the word on the host. Another way to test on the host is to make
the target model closely match the host model. For example, I
currently run a 32-bit little-endian Forth on a 16-bit big-endian
target just so my models match. In the case of the PIC, you could make
the host more closely model the 16-bit target but it sounds to me like
tackling that one would require more Forth experience on your part. My
point about models is that you can think of Forth as a modeling
language instead of a programming language and see where that takes
you.
2. The compiler creates a data structure ADD2 on the host that
contains information like execution address and it also compiles the
definitions into tokens or whatever code the PIC expects to execute.
The host could download this code (ROM image) to the target right away
or it could cache it until you actually test ADD2 or tell the host to
flush the cache.
>
> 1. Instead of taking input from the normal input buffer, I want to
> fundamentally parse a dictionary definition. The way I see it is
> something like running "see <word>" taking the output and parsing it.
> What I'm out of the loop with is accessing dictionary definitions
> directly and parsing the words in those definitions.
Once a definition is compiled, it's a black box. Your code doesn't
have any business peeking inside. SEE is a decompiler (mostly a sanity
checker), which happens to spit out Forth source in some
implementations but assembly code in other implementations.
>
> 2. I know that create puts new words in the dictionary and allot will
> allocate space. I'm unsure how to access that allotted space once I
> created it or exactly how to get a pointer to the definition's alloted
> area. Brad used C, to extend and write into the dictionary space for
> example.
The word >BODY is used to address data structures created by CREATE.
In the case of target compilation, colon would create a data structure
containing the word's execution address along with whatever other
stuff you want. The text interpreter could use >BODY to get that
address and execute it on the target, compile it into the ROM image,
etc.
As an example of a text interpreter, here is a snippet of a text
interpreter that I load on top of an ANS Forth (including GForth),
which I use to compile to a target.
: top-order ( wid -- ) >r get-order nip r> swap set-order ;
: INTERPRETER ( -- ) *INTERPRETER dup top-order set-current ;
: COMPILER ( -- ) *COMPILER dup top-order set-current ;
: T[ ( -- ) *INTERPRETER top-order 0 tstate ! ;
: T] ( -- ) *COMPILER top-order -1 tstate ! ;
: HOST ( -- ) *HOST dup dup 3 set-order *HOST set-current 0
targeting ! ;
: isnumber ( a -- n )
dup dup c@ + c@ [char] . = dup >r if -1 over c+! then \ trim
trailing .
count over c@ [char] - = dup >r if 1 /string then \ accept
leading '-'
0 0 2swap >number abort" ???" drop \ convert
to number
r> if dnegate then
tstate @ if r> if swap ,lit else drop then ,lit
else r> 0= if drop then then ;
: TARGET ( -- ) \ start target interpreter
1 targeting ! 0 targlines !
*HOST *TARGET *TARGET 3 set-order
*TARGET set-current T[
begin targeting @ while
bl word dup c@ if
find dup if tstate @ = >r
dup >body @ magic = \ target word?
if >body cell+ dup
r> if cell+ then @ execute \ ( a -- )
else r> if compile, else execute then
then
else drop isnumber
then
else drop
depth 0< abort" Stack underflow"
source-id if 1 targlines +!
verbose @ if cr targlines @ . .stack then
else ." _ok " .stack cr
then refill 0= abort" Missing HOST directive at end of file"
then
repeat ;
>
> 4. I know that Forth parses and does dictionary lookups. What kinds of
> words should I be looking for to do these activities? Can parsing be
> done from an arbitrary buffer?
>
Parsing should be done from the input buffer, which could be the
console or a file or a text string (see EVALUATE). These are all just
streams of ASCII characters that are generally passed over once,
although some fancy compilers may do more complex things.
I would recommend writing your tools in Forth to get experience
writing Forth. It's a completely different skill from implementing
Forths.
--
Brad Eckert
Brad Eckert
> 1. The compiler creates a word ADD2 on the host that pushes 2 onto the
> target's stack and tells the target to execute the "+" token. That
> tests the word on the host.
Er, it tests it on the target by executing it over the umbilical link.
Then there is the question of what to do with the result. If you leave
it on the target, you can have the text interpreter display the stack
data resident on the target.
Elizabeth D Rather
> In article <1386p75...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
...
>>
>> Yes, this is why I recommended you get a free SwiftX cross-compiler and
>> experiment with it for a while. Write some Forth apps and make them
>> work on some processor SwiftX supports (since the boards are cheap and
>> you aren't developing any new hardware it should be non-traumatic).
>
> Elizabeth,
>
> All the above suggestion does is divert focus away from my real problem.
> You outline 4 things that I don't really need:
>
> 1. A Forth product I don't plan to use.
> 2. Embedded systems hardware I don't plan to buy.
> 3. Applications that I don't need to write.
> 4. A development model that doesn't fit the model I want to use.
>
> As we have discussed elsewhere in this thread, I'm only a novice to the
> Forth language. For programming in general, embedded systems development,
> and the like I know my way around the block.
Getting a little more experience with Forth and Forth cross-compilers
would help you more than you realize.
...
>> The normal way of things is that you load on your host a cross-compiler
>> that looks just like the resident compiler, but generates PIC code. So,
>> no such thing as picompile, just process the source in the same way but
>> you get pic code, ideally automatically downloaded.
>
> I got that from my readings. But it doesn't really address the model of
> testing words on the host, then transferring words to the target. The
> reason is that most forth embedded systems targets are RAM based and
> therefore can be loaded at wire speed. However, in flash based
> environments, the flash write speed often dominates the transfer time.
> In addition flash does have some (and often severe) write cycle
> limitations. So a "test many, write once" model has some merit.
The same model works on flash-based systems. There are obviously
differences in timing, but we have used parts where all code space is in
flash.
It's certainly possible to test your high-level application code on the
host. I teach an advanced Forth course in which students write a
project (traffic light controller) simulated on a PC. In one course,
where the students would be ultimately working with a cross-compiler, I
showed them how to download their completed apps to a 68K target which
they'd never seen and they all worked with very minor changes (they had
to put INTERPRETER before their CREATE ... DOES> defining words, if any,
and TARGET after).
However, it's also possible to use the cross-compiler to test your
PIC-specific code, which can be extremely helpful.
...
>
> What does not seem to be addressed is my specific point of wanting words
> that exist in one environment (HOST) to transfer and compile that word
> on the other side of the fence. In other words I'm looking to avoid the
> following sequence:
>
> HOST
> : add2 2 + ;
> 5 add2 . ( do a bit of testing ) 7 ok
> ( it works so let's get it to the target )
>
> TARGET
> : add2 2 + ( why do I need to retype it? ) ;
>
> I already have the word I want in the host wordset. I don't want to have
> to repeat typing it (and all of its error filled possibilities) in the
> target wordset.
What you're missing is that, once you type in that definition its source
no longer exists, just the version that's compiled and ready for
execution. And what your resident Forth compiled cannot be executed on
the target.
What you need to do is feed your tested source (from keyboard or file)
to a different compiler that will generate PIC code and download it.
You can certainly do this one definition at a time (although a few at a
time would be more convenient), but you can't transfer compiled PC code
to a PIC and expect anything useful to happen.
> Transparency is useful once you have the underlaying system working.
> However, it's not your friend when you're trying to understand how to
> develop the infrastructure you're wanting to implement. That's why I
> motivated my example with an explicit word for the activity.
...
> To reiterate my questions:
>
> 1. Words for accessing dictionary definitions like SEE does?
SEE accesses the compiled version. It isn't hard to do, look at how SEE
is defined in gForth. But I'm not sure it's useful for what you're
trying to do.
> 2. Parsing from somewhere other than the input buffer. Or alternatively
> how to transfer text into the input buffer without typing it?
If you have text somewhere that you wish to parse, just pass it to
EVALUATE (which takes the addr and length of a string and passes it to
the text interpreter).
> 3. words to address allotted memory in the dictionary?
@ and ! and friends access "data space" in Standard Forth. On many
resident systems that's integrated with the dictionary, but in embedded
systems it's usually separate. We have words T@, TC@, T!, and TC! for
target data space, and CT@ etc. for accessing code space. Host @ and !
etc. are switched to use the appropriate target versions during
cross-compilation. There are also versions of ALLOT, etc., for
allocating space in various memory sections. See the XC standard for
details.
Cheers,
Elizabeth
--
none Byron Jeff
>> All the above suggestion does is divert focus away from my real
>> problem. You outline 4 things that I don't really need:
>> 1. A Forth product I don't plan to use.
>> 2. Embedded systems hardware I don't plan to buy.
>> 3. Applications that I don't need to write.
>> 4. A development model that doesn't fit the model I want to use.
>> As we have discussed elsewhere in this thread, I'm only a novice to
>> the Forth language. For programming in general, embedded systems
>> development, and the like I know my way around the block.
>I can sympathise. Sometimes it seems like there's always something else
>to do first before you can do the thing you want.
>But if you want to use Forth, you need to learn how to use Forth. That
>doesn't need to take a long time, but it's a necessary step. So get some
>Forth compiler and practice with it.
I've been noodling with basic constructs. For me the only way to get a
feel for a language is to write a real application in it. This compiler
is the only task that's burning in my brain right. So I'm going to jump
into the deep end of the shark pool and take a swim.
The problem I'm having is that there's a pretty significant disconnect
between tutorial work and the tasks I want to do. I learned more about
what I wanted to pull off from Brad's Writing a Forth Assembler than any
of the three books I studied or the numerous guides/tutorials I've read.
There's a bit of a different skillset doing systems work as opposed to
application work.
>Elizabeth's product isn't the only
>free one that works, but her manual is very good. If you use a different
>one, choose one with a good Forth manual.
I have gforth and picforth which runs on top of it. Both have been fine
for familiarizing myself. The gforth manual is quite extensive and there
are all the books too.
But I'm in a specific area that's outside of the scope of that material.
Hence my questions here.
>Write some sort of
>applications to learn Forth. You learn more by doing than by watching or
>pretending to do. You don't have to use somebody else's cross-compiler
>model but the more of it you can use the less you have to re-invent for
>yourself. That's a plus.
But it's also a minus because you have to get up to speed on what
they've done. What I want to do really isn't too hard:
1. Get the next word from the definition
2. Look up the pic address for the word
3. Generate a call to the pic address.
4. Write the compiled call to the end of the pic compiled definition.
5. Repeat 1-4 for each successive word to the end of the definition.
>
>> >The normal way of things is that you load on your host a
>> >cross-compiler that looks just like the resident compiler, but
>> >generates PIC code. So, no such thing as picompile, just process the
>> >source in the same way but you get pic code, ideally automatically
>> >downloaded.
>>
>> I got that from my readings. But it doesn't really address the model
>> of testing words on the host, then transferring words to the target.
>> The reason is that most forth embedded systems targets are RAM based
>> and therefore can be loaded at wire speed. However, in flash based
>> environments, the flash write speed often dominates the transfer time.
>> In addition flash does have some (and often severe) write cycle
>> limitations. So a "test many, write once" model has some merit.
>
>I see! So you might want to write and test a whole block of code with
>your simulator, and then when you can fill a whole block of your flash
>then transfer them all at once.
Something like that. Incremental development with a variable number of
definitions per increment.
Advanced work would be a distributed model where the PIC and host work
together to run an application.
>On the host you'd still want to keep track of the addresses to call on
>the PIC so you could test them easily.
Exactly. See above. All the code on the target would be headerless. All
names and addresses for routines would be on the host.
>It looks like the concession
>you're making to reality here is that you want to flash blocks at a time
>and do your final testing for whole blocks of words at once, in the hope
>that you'll only have to do it again a few times.
Right. So test the word(s) on the host and do a limited number of writes
to the target. It's antithetical to the forth way because generally
forth machines operate from RAM.
Now that's a thought. You'd need some kind of versioning so that only
new and updated words are actually update.
I'll go read up on file I/O. Thanks.
>> : add2 2 + ;
>> : add4 add2 add2 ;
>> 5 add4 . 9 ok
>>
>> Being happy with the result, let's get the word into target space:
>>
>> add4 picompile ok
>
>You can do that. The easy way, you do
>
>' add2 picompile
>' add4 picompile
Thanks for that too. The ' operates one word forward. Got it.
I'm aware that adding embedded words by hand is an option. The picompile
command could simply fail on an unknown word. But if it can go ahead and
compile that word in situ it would be a good thing. Seamless.
>so you don't have to look up each command it calls to make sure they're
>already compiled.
>You want your code to use relative jumps and relative calls, right?
Nope. PIC's don't have relative calls. It's absolute calls and most
likely absolute jumps too.
>So
>in the best case you can just take your block of code and pass it to the
>PIC and flash it. That shouldn't be too hard.
I don't think it would be. That's the end of the process.
>> Now add4 refers to a word (add2) that's currently not compiled in the
>> target space. It would be nice to have a transparent compilation of
>> that additional word into the target space too.
>
>You can do that too but it's a little harder. You want to compile each
>of the words it calls, recursively. You want to recurse your way down to
>a word that doesn't call anything new and compile that before you start
>compiling anything else. I think you'll probably have to give up
>single-pass compilation to do that, but maybe not. It's potentially a
>lot of complication compared to just checking for yourself. If it was me
>doing it, I'd think about how much code I was going to write that needed
>it, and I'd probably put off doing that step until I noticed I was
>making errors without it. If you put your time into whatever looks like
>the limiting factor now, it's likely to give you faster progress.
It's an advanced feature. For a first pass, simply failing on an
uncompiled word will be fine.
>> 1. Words for accessing dictionary definitions like SEE does?
>
>Like SEE ? Ah. You get an xt, or anyway the address of a call, and you
>want to find the name. The newer Forth standards don't give you any way
>to do that since some Forth systems can't do it easily. The Forth-83
>standard let you do that, I think. There was a word like >NAME . I don't
>remember whether you had to do >BODY >NAME or some other combination.
>You might check the documentation of your particular Forth, or look at
>the code with DUMP or something like that to see how it's actually laid
>out.
Not standard. Got it. I'll figure out how gforth does it then.
>> 2. Parsing from somewhere other than the input buffer. Or
>> alternatively how to transfer text into the input buffer without
>> typing it?
>
>EVALUATE . or LOAD for up to 1024 characters.
Will take a look.
>> 3. words to address allotted memory in the dictionary?
>
>HERE 256 cells allot CONSTANT MY-DATA
>ok
>MY-DATA U.
>ok 21063284
>
>Give it the name to get the base address. Add whatever offset you want
>from there to get later addresses.
>
>CREATE MY-DATA2 32 CELLS ALLOT
>MY-DATA2 acts just like MY-DATA except the code gets compiled first.
Excellent. and I can use ! and @ to access that memory.
>When the one thing that's most getting in your way is that you don't
>know Forth, it looks like learning Forth would be the limiting factor
>that deserves some effort.
I am certainly working on it. It's just that for the tasks that I need
to do, there isn't a generic book that describes it. I need the book
"Using Forth for Systems Programming."
Anyone have a copy?
BAJ
none Byron Jeff
Elizabeth D Rather <erath...@forth.com> wrote:
>none Byron Jeff wrote:
>> In article <1386p75...@news.supernews.com>,
>> Elizabeth D Rather <erath...@forth.com> wrote:
>...
>>>
>>> Yes, this is why I recommended you get a free SwiftX cross-compiler and
>>> experiment with it for a while. Write some Forth apps and make them
>>> work on some processor SwiftX supports (since the boards are cheap and
>>> you aren't developing any new hardware it should be non-traumatic).
>>
>> Elizabeth,
>>
>> All the above suggestion does is divert focus away from my real problem.
>> You outline 4 things that I don't really need:
>>
>> 1. A Forth product I don't plan to use.
>> 2. Embedded systems hardware I don't plan to buy.
>> 3. Applications that I don't need to write.
>> 4. A development model that doesn't fit the model I want to use.
>>
>> As we have discussed elsewhere in this thread, I'm only a novice to the
>> Forth language. For programming in general, embedded systems development,
>> and the like I know my way around the block.
>
>Getting a little more experience with Forth and Forth cross-compilers
>would help you more than you realize.
Until I get this going I'll be working with gforth and picforth. My
interns and I do have projects that we need to get done.
>...
>>> The normal way of things is that you load on your host a cross-compiler
>>> that looks just like the resident compiler, but generates PIC code. So,
>>> no such thing as picompile, just process the source in the same way but
>>> you get pic code, ideally automatically downloaded.
>>
>> I got that from my readings. But it doesn't really address the model of
>> testing words on the host, then transferring words to the target. The
>> reason is that most forth embedded systems targets are RAM based and
>> therefore can be loaded at wire speed. However, in flash based
>> environments, the flash write speed often dominates the transfer time.
>> In addition flash does have some (and often severe) write cycle
>> limitations. So a "test many, write once" model has some merit.
>
>The same model works on flash-based systems. There are obviously
>differences in timing, but we have used parts where all code space is in
>flash.
>
>It's certainly possible to test your high-level application code on the
>host. I teach an advanced Forth course in which students write a
>project (traffic light controller) simulated on a PC. In one course,
>where the students would be ultimately working with a cross-compiler, I
>showed them how to download their completed apps to a 68K target which
>they'd never seen and they all worked with very minor changes (they had
>to put INTERPRETER before their CREATE ... DOES> defining words, if any,
>and TARGET after).
That's closer to the model I'm after. Truly I want a seamless blended
environment where there's no distinction between the target and the
host. What I'm proposing is a first step to get there.
>However, it's also possible to use the cross-compiler to test your
>PIC-specific code, which can be extremely helpful.
>
>...
>>
>> What does not seem to be addressed is my specific point of wanting words
>> that exist in one environment (HOST) to transfer and compile that word
>> on the other side of the fence. In other words I'm looking to avoid the
>> following sequence:
>>
>> HOST
>> : add2 2 + ;
>> 5 add2 . ( do a bit of testing ) 7 ok
>> ( it works so let's get it to the target )
>>
>> TARGET
>> : add2 2 + ( why do I need to retype it? ) ;
>>
>> I already have the word I want in the host wordset. I don't want to have
>> to repeat typing it (and all of its error filled possibilities) in the
>> target wordset.
>
>What you're missing is that, once you type in that definition its source
>no longer exists, just the version that's compiled and ready for
>execution.
I do understand that. What I don't understand is how words like SEE do
their work. SEE has to backtrace the XTs in the compiled word back to
the names that the XTs map to. How does that happen?
>And what your resident Forth compiled cannot be executed on
>the target.
I don't want to execute that compiled code on the target. I want to get
the word names that represented in the definition and then take those
names and recompile them for the target. In gforth (or any other for
that matter) how does the following sequence work:
: add2 2 + ; ok
see add2
: add2
2 + ; ok
The first line clearly compiles the word add2 on the host. The SEE
command displays the names of the words of the definition. It decompiles
the definition. How does that work?
>What you need to do is feed your tested source (from keyboard or file)
>to a different compiler that will generate PIC code and download it.
That's the traditional way of doing it. I'm searching for a different
model.
>You can certainly do this one definition at a time (although a few at a
>time would be more convenient), but you can't transfer compiled PC code
>to a PIC and expect anything useful to happen.
Nor do I. I want the NAMES in the compiled host definition, not the XTs
or the compiled code. See gives the names of the words. I just want to
programmatically get that back and feed it to the parser of the cross
compiler.
>> Transparency is useful once you have the underlaying system working.
>> However, it's not your friend when you're trying to understand how to
>> develop the infrastructure you're wanting to implement. That's why I
>> motivated my example with an explicit word for the activity.
>
>...
>> To reiterate my questions:
>>
>> 1. Words for accessing dictionary definitions like SEE does?
>
>SEE accesses the compiled version. It isn't hard to do, look at how SEE
>is defined in gForth. But I'm not sure it's useful for what you're
>trying to do.
See above.
>> 2. Parsing from somewhere other than the input buffer. Or alternatively
>> how to transfer text into the input buffer without typing it?
>
>If you have text somewhere that you wish to parse, just pass it to
>EVALUATE (which takes the addr and length of a string and passes it to
>the text interpreter).
Excellent! Will take a look.
>> 3. words to address allotted memory in the dictionary?
>
>@ and ! and friends access "data space" in Standard Forth. On many
>resident systems that's integrated with the dictionary, but in embedded
>systems it's usually separate. We have words T@, TC@, T!, and TC! for
>target data space, and CT@ etc. for accessing code space. Host @ and !
>etc. are switched to use the appropriate target versions during
>cross-compilation. There are also versions of ALLOT, etc., for
>allocating space in various memory sections. See the XC standard for
>details.
I can see the variations. All the accesses I want are on the host. So
it's probably integrated. I'll work with the examples in the other post
to see how to get it done.
Thanks for the help.
BAJ
Elizabeth D Rather
> In article <1387v9j...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>>> As we have discussed elsewhere in this thread, I'm only a novice to the
>>> Forth language. For programming in general, embedded systems development,
>>> and the like I know my way around the block.
>> Getting a little more experience with Forth and Forth cross-compilers
>> would help you more than you realize.
>
> Until I get this going I'll be working with gforth and picforth. My
> interns and I do have projects that we need to get done.
You're asking really basic questions about words like ' and EVALUATE.
As several others have indicated, you really need to develop basic Forth
skills to do anything at all, especially the kind of projects you're
planning.
...
>> What you're missing is that, once you type in that definition its source
>> no longer exists, just the version that's compiled and ready for
>> execution.
>
> I do understand that. What I don't understand is how words like SEE do
> their work. SEE has to backtrace the XTs in the compiled word back to
> the names that the XTs map to. How does that happen?
SEE doesn't necessarily reconstruct the source, at best it can show you
what was compiled. It's easier on ITC implementations, where you have
strings of xt's that you can look up. On optimized direct code
implementations it's impossible to reconstruct the source, because the
optimized code is completely different.
LOCATE displays source, by going back to the file from which it was
generated. It can be impossible to accurately reconstruct a word typed
from the keyboard.
>> And what your resident Forth compiled cannot be executed on
>> the target.
>
> I don't want to execute that compiled code on the target. I want to get
> the word names that represented in the definition and then take those
> names and recompile them for the target. In gforth (or any other for
> that matter) how does the following sequence work:
>
> : add2 2 + ; ok
>
> see add2
> : add2
> 2 + ; ok
>
> The first line clearly compiles the word add2 on the host. The SEE
> command displays the names of the words of the definition. It decompiles
> the definition. How does that work?
As Jonah says, on some systems you can get to the name field from the xt
or CALL destination. However, neither the ability to do that nor the
arrangement of the name field itself is standardized. Consult your
gForth docs to see how they do it. But first look at some words using
structures (e.g. IF ... THEN) to see how SEE represents them on gForth
and whether that will work for you.
none Byron Jeff
Elizabeth D Rather <erath...@forth.com> wrote:
>none Byron Jeff wrote:
>> In article <1387v9j...@news.supernews.com>,
>> Elizabeth D Rather <erath...@forth.com> wrote:
>...
>>>> As we have discussed elsewhere in this thread, I'm only a novice to the
>>>> Forth language. For programming in general, embedded systems development,
>>>> and the like I know my way around the block.
>>> Getting a little more experience with Forth and Forth cross-compilers
>>> would help you more than you realize.
>>
>> Until I get this going I'll be working with gforth and picforth. My
>> interns and I do have projects that we need to get done.
>
>You're asking really basic questions about words like ' and EVALUATE.
I am. I've read through Starting Forth, Thinking Forth, and Programming
Forth. Only Stephen's book has even the briefest mention of EVALUATE.
I need a reference that is both a Rosetta Stone to more traditional
languages and outlines features unique to Forth. I read through a couple
of thousand posts going back to 2003 in this newsgroup before posting my
first post. I know it seems elementary to you, but you've been at this
for over 30 years.
>As several others have indicated, you really need to develop basic Forth
>skills to do anything at all, especially the kind of projects you're
>planning.
I need a reference. I read what I thought were definitive Forth
references. But they focus on the baby steps required for any language.
So the tendency is to gloss over them. A IF statement is an IF statement
in virtually any language. While Forth's repetition constructs with
BEGIN, WHILE, REPEAT, AGAIN are slightly complex, they are only slightly
complex.
What I'm asking for isn't basic forth. If it were I would have already
located it.
The only referred to book I haven't read are your two. I'll make a point
of reading the PDF in the SwiftX download in the next couple of days.
>
>...
>>> What you're missing is that, once you type in that definition its source
>>> no longer exists, just the version that's compiled and ready for
>>> execution.
>>
>> I do understand that. What I don't understand is how words like SEE do
>> their work. SEE has to backtrace the XTs in the compiled word back to
>> the names that the XTs map to. How does that happen?
>
>SEE doesn't necessarily reconstruct the source, at best it can show you
>what was compiled. It's easier on ITC implementations, where you have
>strings of xt's that you can look up. On optimized direct code
>implementations it's impossible to reconstruct the source, because the
>optimized code is completely different.
So SEE isn't universal? That makes sense. I happen to have gforth, which
seems to reconstruct all non code (and just testing, code...) words.
Truthfully I don't need to know how SEE does its job. All I need is to
be able to invoke it and capture its output for further processing.
Is there a Forth reference that maps "I need to do task X, use word Y".
I'd be happy to use it.
BAJ
Jonah Thomas
> Jonah Thomas <j2th...@cavtel.net> wrote:
> >byron@upstairs.(none) (Byron Jeff) wrote:
> >> As we have discussed elsewhere in this thread, I'm only a novice to
> >> the Forth language. For programming in general, embedded systems
> >> development, and the like I know my way around the block.
> >But if you want to use Forth, you need to learn how to use Forth.
> >That doesn't need to take a long time, but it's a necessary step. So
> >get some Forth compiler and practice with it.
>
> I've been noodling with basic constructs. For me the only way to get a
> feel for a language is to write a real application in it. This
> compiler is the only task that's burning in my brain right. So I'm
> going to jump into the deep end of the shark pool and take a swim.
OK. It sounds like a good project. Maybe you can break it into little
pieces and start with the ones you know how to do.
> The problem I'm having is that there's a pretty significant disconnect
> between tutorial work and the tasks I want to do. I learned more about
> what I wanted to pull off from Brad's Writing a Forth Assembler than
> any of the three books I studied or the numerous guides/tutorials I've
> read. There's a bit of a different skillset doing systems work as
> opposed to application work.
In Forth, pretty much everything you're talking about is advanced
application work. Some of it is basic applicsation work.
> 1. Get the next word from the definition
> 2. Look up the pic address for the word
> 3. Generate a call to the pic address.
> 4. Write the compiled call to the end of the pic compiled definition.
> 5. Repeat 1-4 for each successive word to the end of the definition.
OK. You want to have a variable that holds the first available code
address on the PIC . When you compile a command, you make a definition
on the host that returns that address. You also make a second definition
that puts PIC code on the host for the simulator to execute. Whenever
you compile more code into the simulated definition, you increment the
variable to remind you about the PIC memory you're using up.
So you can compile source code, and each word tells you the PIC address.
Making the call to that address and writing the code to the end of the
current definition ought to be easy.
Your simulator has to use PIC addresses and convert them to real
addresses to call, but that's easy.
> >It looks like the concession
> >you're making to reality here is that you want to flash blocks at a
> >time and do your final testing for whole blocks of words at once, in
> >the hope that you'll only have to do it again a few times.
>
> Right. So test the word(s) on the host and do a limited number of
> writes to the target. It's antithetical to the forth way because
> generally forth machines operate from RAM.
It looks to me like a slight complication.
> >> HOST
> >> : add2 2 + ;
> >> 5 add2 . ( do a bit of testing ) 7 ok
> >> ( it works so let's get it to the target )
> >>
> >> TARGET
> >> : add2 2 + ( why do I need to retype it? ) ;
> >>
> >> I already have the word I want in the host wordset. I don't want to
> >> have to repeat typing it (and all of its error filled
> >possibilities)> in the target wordset.
> >
> >One way is to put the word in a file, and load the file. And then you
> >might as well keep your tests in a file too.
>
> Now that's a thought. You'd need some kind of versioning so that only
> new and updated words are actually update.
>
> I'll go read up on file I/O. Thanks.
You have HOST SIMULATE and TARGET . HOST compiles in Forth on the host,
and you can use it to test your logic. SIMULATE compiles PIC code on the
host. There's no particular reason to decompile the Forth code to
reconstitute your source and recompile it as PIC unless you really want
to. Probably simpler just to recompile the source. TARGET compiles on
the PIC and if you designed your simulator so it runs exactly the same
code that will run on the PIC then you might want to just dump that code
to the PIC to put it in flash.
> >' add2 picompile
> >' add4 picompile
>
> Thanks for that too. The ' operates one word forward. Got it.
That's an early-intermediate application-code thing.
> >> 1. Words for accessing dictionary definitions like SEE does?
> >
> >Like SEE ? Ah. You get an xt, or anyway the address of a call, and
> >you want to find the name. The newer Forth standards don't give you
> >any way to do that since some Forth systems can't do it easily. The
> >Forth-83 standard let you do that, I think. There was a word like
> >>NAME . I don't remember whether you had to do >BODY >NAME or some
> >other combination. You might check the documentation of your
> >particular Forth, or look at the code with DUMP or something like
> >that to see how it's actually laid out.
>
> Not standard. Got it. I'll figure out how gforth does it then.
You could try SEE SEE . You might get something by SEE WORDS too.
Remember, your point in doing this is to decompile code so you can
recompile it. If you re-use the source code you can skip this whole
step.
> >> 3. words to address allotted memory in the dictionary?
> >
> >HERE 256 cells allot CONSTANT MY-DATA
> >ok
> >MY-DATA U.
> >ok 21063284
> >
> >Give it the name to get the base address. Add whatever offset you
> >want from there to get later addresses.
> >
> >CREATE MY-DATA2 32 CELLS ALLOT
> >MY-DATA2 acts just like MY-DATA except the code gets compiled first.
>
> Excellent. and I can use ! and @ to access that memory.
Yes, this is pretty basic application stuff.
> >When the one thing that's most getting in your way is that you don't
> >know Forth, it looks like learning Forth would be the limiting factor
> >that deserves some effort.
>
> I am certainly working on it. It's just that for the tasks that I need
> to do, there isn't a generic book that describes it. I need the book
> "Using Forth for Systems Programming."
I didn't think about this stuff being advanced topics, but I guess maybe
it is. So what you need is
parsing words
dictionary words
compiling words
data structures
These are likely to do most of what you want.
Parsing words look ahead in the input buffer. PARSE WORD PARSE-WORD (or
PARSE-NAME ) SKIP will let you build most of what you'd want, if you
don't mind doing things one line at a time. You might find yourself
re-inventing standard words that you happen not to know the names of.
Dictionary words. FIND and SEARCH-WORDLIST find words. ' or ['] finds a
word but it bombs out if the word isn't there. Only use that when you
know for sure the word is defined. WORDLIST makes a new empty dictionary
you can add to. SET-CURRENT and GET-CURRENT deal with the default
dictionary. (Forthers call all the dictionaries together "the
dictionary" and call individual ones vocabularies or wordlists.)
GET-ORDER and SET-ORDER deal with which wordlists get searched by
default and in which order. MARKER lets you get rid of the latest
definitions.
compiling words. IMMEDIATE makes a word execute during compilation so it
will help the compiler do things instead of just get compiled itself.
DOES> is worth special attention. RECURSE Probably some other words I
DOES> haven't thought of would be extra useful.
Data structures. CREATE DOES> (You probably don't want ALLOCATE ).
HERE , C, ALLOT
I think if you figure out how to use these they'll give you most of the
tools you need to build whatever you want. Again, you might redesign
some wheels that are universally available, but if it's easier to
rewrite them than to find out about them then that's the way to go.
Stephen Pelc
wrote:
>What does not seem to be addressed is my specific point of wanting words
>that exist in one environment (HOST) to transfer and compile that word
>on the other side of the fence. In other words I'm looking to avoid the
>following sequence:
>
>HOST
>: add2 2 + ;
>5 add2 . ( do a bit of testing ) 7 ok
>( it works so let's get it to the target )
>
>TARGET
>: add2 2 + ( why do I need to retype it? ) ;
You don't need to retype it! A proper cross compiler will allow:
: add2 2 + ;
5 add2 . ( do a bit of testing ) 7 ok
That's all. The complexity is in managing the Flash - on some
processors Flash management is a real pain.
>To reiterate my questions:
>
>1. Words for accessing dictionary definitions like SEE does?
>2. Parsing from somewhere other than the input buffer. Or alternatively
>how to transfer text into the input buffer without typing it?
>3. words to address allotted memory in the dictionary?
These are facilities provided by the vendors already. It's
a fact that the commercial cross compilers are just much more
evolved than the others, which are mostly designed as point
tools. Before you sit down to write both a Forth kernel and
a cross compiler, you really should write a non-trivial Forth
application yourself. Forth is subtle, and cross compilers,
with all their different time-frames to consider, are even more
subtle.
Stephen
--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
Elizabeth D Rather
> In article <13886l9...@news.supernews.com>,
...
>> You're asking really basic questions about words like ' and EVALUATE.
>
> I am. I've read through Starting Forth, Thinking Forth, and Programming
> Forth. Only Stephen's book has even the briefest mention of EVALUATE.
Starting Forth is way too elementary, and grossly out of date. Thinking
Forth is good in its way, but not very technical. Actually, I'm
currently polishing a new edition of my book, Forth Programmer's
Handbook (vastly improved, IMO). Send me your email and I'll get you a
draft (no, not everybody, just Byron right now). I'm erather {at} forth
{dot} com.
...
> What I'm asking for isn't basic forth. If it were I would have already
> located it.
>
> The only referred to book I haven't read are your two. I'll make a point
> of reading the PDF in the SwiftX download in the next couple of days.
See above. Handbook is a lot more technical and comprehensive than
Starting Forth and Thinking Forth.
...
> So SEE isn't universal? That makes sense. I happen to have gforth, which
> seems to reconstruct all non code (and just testing, code...) words.
>
> Truthfully I don't need to know how SEE does its job. All I need is to
> be able to invoke it and capture its output for further processing.
Actually, it would probably be very instructive for you to look at how
SEE does its job. As someone else suggested, try SEE SEE.
> Is there a Forth reference that maps "I need to do task X, use word Y".
> I'd be happy to use it.
The problem is, there's a virtually unlimited number of 'task X'
references. And there really is a different approach to these issues in
Forth; folks who approach with the attitude, "ok, I know how to do this
in C (or Basic, or whatever), what's the Forth equivalent?" often end up
writing very bad Forth. This is why reading existing Forth code (such as
SEE) can be instructive. gForth comes with source, doesn't it?
At a fairly high level, this is what I attempted to address in my "Forth
Application Techniques" book.
Frank Buss
> I need a reference. I read what I thought were definitive Forth
> references. But they focus on the baby steps required for any language.
This is the definitive Forth reference:
http://www.taygeta.com/forth/dpans.htm
In Win32Forth (and maybe other implementations) it is included in the
system and you can enter e.g. "help evaluate" to jump to the definition of
a word (if you've loaded the helpsystem with "help-system" before).
But maybe it is not the Rosetta Stone you are searching for, but then
you'll need it anyway if you want to write your own Forth compiler. And the
more Forth you know, the more of the ANS Standard document you'll
understand :-)
You can learn Forth only by programming Forth. After the first basic steps,
reading books doesn't help very much. Reading good other sources, like the
samples, libraries and implementation of Forth systems, helps a bit.
none Byron Jeff
>> In article <13886l9...@news.supernews.com>,
>...
>>> You're asking really basic questions about words like ' and EVALUATE.
>>
>> I am. I've read through Starting Forth, Thinking Forth, and Programming
>> Forth. Only Stephen's book has even the briefest mention of EVALUATE.
>
>Starting Forth is way too elementary, and grossly out of date. Thinking
>Forth is good in its way, but not very technical. Actually, I'm
>currently polishing a new edition of my book, Forth Programmer's
>Handbook (vastly improved, IMO). Send me your email and I'll get you a
>draft (no, not everybody, just Byron right now). I'm erather {at} forth
>{dot} com.
Thanks for the offer. E-mail sent to you offline.
>
>...
>> What I'm asking for isn't basic forth. If it were I would have already
>> located it.
>>
>> The only referred to book I haven't read are your two. I'll make a point
>> of reading the PDF in the SwiftX download in the next couple of days.
>
>See above. Handbook is a lot more technical and comprehensive than
>Starting Forth and Thinking Forth.
That will be helpful.
>
>...
>> So SEE isn't universal? That makes sense. I happen to have gforth, which
>> seems to reconstruct all non code (and just testing, code...) words.
>>
>> Truthfully I don't need to know how SEE does its job. All I need is to
>> be able to invoke it and capture its output for further processing.
>
>Actually, it would probably be very instructive for you to look at how
>SEE does its job. As someone else suggested, try SEE SEE.
Here's the output:
SEE SEE
: see
parse-word find-name dup 0=
IF drop -13 throw
THEN
name-see ;
It's the only problem I see (pun intended I guess) with a generative
language. Words at the top of the pyramid per-se are supported by a
collection of words with functions that can only be intuited by name.
However I did stumble upon a somewhat helpful technique. Dropping a word
along with the name gforth yielded a pointer into the gforth manual on
the subject. That should be helpful in moving forward.
I really need to read the gforth manual more deeply. It has insight.
>> Is there a Forth reference that maps "I need to do task X, use word Y".
>> I'd be happy to use it.
>
>The problem is, there's a virtually unlimited number of 'task X'
>references. And there really is a different approach to these issues in
>Forth; folks who approach with the attitude, "ok, I know how to do this
>in C (or Basic, or whatever), what's the Forth equivalent?" often end up
>writing very bad Forth. This is why reading existing Forth code (such as
>SEE) can be instructive. gForth comes with source, doesn't it?
It does. But I think the documentation is more helpful than the
decomposition. I'll take the time to peruse section 5 of the gForth
manual labeled "Forth words".
>At a fairly high level, this is what I attempted to address in my "Forth
>Application Techniques" book.
Yet something else I need to read.
Thanks for the pointers.
BAJ
none Byron Jeff
Stephen Pelc <steph...@INVALID.mpeforth.com> wrote:
>On Thu, 28 Jun 2007 06:44:34 -0500, byron@upstairs.(none) (Byron Jeff)
>wrote:
>
>>What does not seem to be addressed is my specific point of wanting words
>>that exist in one environment (HOST) to transfer and compile that word
>>on the other side of the fence. In other words I'm looking to avoid the
>>following sequence:
>>
>>HOST
>>: add2 2 + ;
>>5 add2 . ( do a bit of testing ) 7 ok
>>( it works so let's get it to the target )
>>
>>TARGET
>>: add2 2 + ( why do I need to retype it? ) ;
>
>You don't need to retype it! A proper cross compiler will allow:
>
>: add2 2 + ;
>5 add2 . ( do a bit of testing ) 7 ok
Don't you need a TARGET in front of that definition? ;-)
>That's all. The complexity is in managing the Flash - on some
>processors Flash management is a real pain.
As with my target. Enough of a pain that I want to test on the host
before committing to the target.
Umbilical cross-compiling Forths are great for RAM based targets. It's a
virtually transparent load and go, giving the impression of an
interactive environment due to what's essentially JIT compiling and
loading of new definitions.
Flash is slow to load. Flash wears out (really quickly with some devices)
So a test/commit type cycle has merit in those environments.
>
>>To reiterate my questions:
>>
>>1. Words for accessing dictionary definitions like SEE does?
>>2. Parsing from somewhere other than the input buffer. Or alternatively
>>how to transfer text into the input buffer without typing it?
>>3. words to address allotted memory in the dictionary?
>
>These are facilities provided by the vendors already. It's
>a fact that the commercial cross compilers are just much more
>evolved than the others, which are mostly designed as point
>tools. Before you sit down to write both a Forth kernel and
>a cross compiler, you really should write a non-trivial Forth
>application yourself. Forth is subtle, and cross compilers,
>with all their different time-frames to consider, are even more
>subtle.
I may fish around for something else to work on. Both my interns have
tasks to work on in forth (Vending machine interface and PIC flash
bootloader). I may find some insight there.
BAJ
klp...@cox.net
[snip]
BAJ, have you seen http://pic18forth.sourceforge.net/ ?
none Byron Jeff
Jonah Thomas <j2th...@cavtel.net> wrote:
>byron@upstairs.(none) (Byron Jeff) wrote:
>
>> Problem Definition: I want to develop an incremental word cross
>> compiler for the PIC. I hope this has been a "been there, done that."
>> type exercise for some of you.
>Yes, it is.
Cool.
>> The idea is to develop and test the word on
>> the host, then cross compile the word into its PIC equivalent for
>> transport to the target. So for example say I develop a simple word on
>> the host:
>>
>> : add2 2 + ;
>>
>> I test it out to see that it works:
>>
>> 5 add2 . 7 ok
>>
>> Now I'd like to put the word permanently on the pic.
>
>Say you have a Forth interpreter/compiler that can both run Forth, and
>it also has a PIC assembler and a way to send code to the PIC.
OK.
>Ideally you'd set it up so you can just switch back and forth. Do FORTH
>and it compiles things into the host and interprets things on the host.
>5 add2 . will first put 5 on the host stack, add2 turns it to 7, and .
>displays it.
Testing on the host is desirable.
>Do PIC and now 5 sends a 5 to the PIC's stack, add2 will
>look up the address of the add2 command that's compiled on the PIC and
>tells the PIC to execute that, and . tells the PIC to send the TOS back
>to the host to be displayed.
I get the hybrid. Eventually I'd like to get there. What's still murky
is how to get the PIC to a point where it can be completely untethered
from the host in this instance.
Also when in this process is add2 compiled and loaded on the PIC. Is it
JIT, or does it have to be preloaded before executing
PIC 5 add2 .
?
>Doesn't that seem like a sort of ideal for ease-of-use?
It's a model that close to the one I'm seeking. I don't mind having to
manually specify which words to transfer to the target. Actually I'd
prefer it.
>You might want a
>few more commands available. but that gives you most of it. Switch to
>PIC and do : FOO and you add a name FOO to the host dictionary, and
>start compiling a routine on the PIC, and get the address of the start
>of the routine to put into FOO on the host.
I follow up to a point. It falls down for me with the fact that FOO has
most likely already been written and tested as a word in the host space
and I'd like to be able to recompile it and transfer into PIC space.
>Then each PIC routine you
>access gets added to the definition on the PIC until you get to ; which
>compiles an exit on the PIC .
Again I follow. I'd just really like to redirect where the source of
words that are compiled are coming from.
>So you need a few special commands. : IF THEN ; etc. Things that do
>special compilation tricks etc, that you can and should some of the
>calculation on the host and then add the result to the PIC .
>
>If you look at the cross-compiler standard, it will give you suggested
>solutions to all of the problems. Or at least, all the ones that aren't
>so well-known in Forth that they forgot to mention them. You don't have
>to do it their way, but it's probably easier to see what they're doing
>than figure it all out from first principles.
WHat I believe I'll get from those standards is how the transition words
(HOST, TARGET, INTERPRETER, etc) actually switch the environments
seamlessly.
>> Questions:
>>
>> 1. Instead of taking input from the normal input buffer, I want to
>> fundamentally parse a dictionary definition. The way I see it is
>> something like running "see <word>" taking the output and parsing it.
>> What I'm out of the loop with is accessing dictionary definitions
>> directly and parsing the words in those definitions.
>
>If you have a definition in a string, with most Forths you can give the
>address and length of the string and then do EVALUATE and it will parse
>the string. If the definition is in a series of strings with line breaks
>you can EVALUATE them one at a time. Since it's a single-pass evaluation
>you don't have to worry about going back and picking up things from an
>old buffer. Single-pass evaluation avoids so many problems I strongly
>suggest you avoid multi-pass evaluation for your special PIC compiler.
Got it. Will test.
>> 2. I know that create puts new words in the dictionary and allot will
>> allocate space. I'm unsure how to access that allotted space once I
>> created it or exactly how to get a pointer to the definition's alloted
>> area. Brad used C, to extend and write into the dictionary space for
>> example.
>
>CREATE FOO 500 CELLS ALLOT
>
>FOO . will give you the address of the beginning of the buffer. It's
>like FOO is a variable, it gives you the address, but instead of one
>cell you can allot as much space as you want. Was that what you wanted,
>or were you asking how to implement that?
It's part of what I needed. I'll go and read up on some array
implementations for the rest.
>> 3. I used strings above due to ignorance. How could one simplify the
>> examples to
>>
>> add2 picompile
>> add2 picload
>
>You want to compile the definition on the host first, test it, and then
>compile it onto the PIC.
Bingo!
>Or maybe you'd like to compile PIC code onto
>the host and test it with a simulator and then download it to the PIC.
Nope. I want to test Forth, not PIC assembly.
>I'd suggest the first way first -- if there's a problem with your logic,
>fix that before you get involved with whether your simulator or your
>assembler have a problem.
Right. I'm looking to add the host as a forth modelling engine in
addition to its cross-compiling, dictionary managing, and umbilical
mangaging duties.
>So FORTH to compile and test the definition in Forth.
Yes.
> Then SIM to compile and test the definition on the simulator.
Maybe. If the underlaying kernel on the PIC is solid and the tether in
place, this could be skipped.
>Then PIC to compile and test the definition on the PIC .
Yes. But compiling from the existing host forth definition instead of
having to type it over, or possibly having to get it from a file.
I may have to rethink that last bit simply because it's a bit too
dynamic. The host is going to have to have the ability to save and
reload the PIC environment for both development and maintenance. But
personally I'd perfer a LOAD/SAVE interface analogous to an editing
session as opposed to a EDIT/COMPILE/DOWNLOAD model which is in the more
traditional development style. I mean an editor doesn't write back to
the file on disk after every change, so why should my system have to
write to the PIC flash everytime a new word is added.
So to completely capture the state of the PIC environment each
definition would need three components:
1) Source line for definition
2) Address of definition on target PIC. Use a sentinel (like HEX FFFF)
to indicate that this definition isn't current written to PIC flash.
3) Definition compiled for the PIC, though technically there's no need
to keep that compiled definition around once it's compiled and loaded
into the PIC flash. Plus it can always be recompiled from the source
definition if necessary.
Now treat the forth execution run on the host like an editing session.
First you load the PIC environment from a file. definitions (since they
are now text source) can be compiled and tested on the host. Committing
the environment compiles any new words for the PIC and downloads them to
the target along with updating the definition addresses on the host.
When finished with the session, save the environment back to a file for
the next time.
>Now, you want to use the name of the definition to point to the
>definition so you can do things with it?
That's what I was originally thinking. Elizabeth has explained that the
decompiling feature of the SEE word really has no guarantees. So I'm
starting to think now that actually capturing the source text of the
definition may be a better win for several reasons. That source text can
be saved in a file. It can be compiled for the both the host and the
target without requiring a dependancy between the two. The cool thing
about Forth is that literally the : word can be redefined to manage all
of this.
>Or point to the code so you
>don't have to recompile the PIC code for the PIC after you've already
>done it once for the simulator?
The simulator isn't as big a win as you think. There's no substative
reason to believe that if the logic of a word works correctly on the
host, that you would not get similar results on the target. There
probably isn't a huge reason to go from host->sim->target as opposed to
host->target directly.
>You can do that. Or you could make a
>file named add2.txt and take it from there, or really whatever you like.
Too many choices. That's one of my problems.
>> 4. I know that Forth parses and does dictionary lookups. What kinds of
>> words should I be looking for to do these activities? Can parsing be
>> done from an arbitrary buffer?
>
>Yes, EVALUATE parses from an arbitrary buffer.
Thanks for that. I'm glad I asked the question. Now I know where to
look.
>You can look up one word with FIND or, differently, the more modern word
>SEARCH-WORDLIST .
>You can parse one word from the current input buffer with WORD or the
>more modern word PARSE-NAME (or PARSE-WORD ).
Both of these are helpful.
>But look, you're asking very basic Forth questions. If you're going to
>use Forth you might as well figure out how. If you asked a whole lot of
>questions that were in the FAQ at some point people would start pointing
>you to the FAQ . And it really makes sense to point you to a Forth
>manual.
I'm going to take a deep read of the forth words implemented in gForth
manual. I think it'll give me a better idea of what's going on.
>Maybe you could start some simple project in Forth to get clearer about
>the language. Lots of people write a simple Forth compiler to learn the
>language, which unfortunately tends to give them more experience at
>writing Forth compilers than at using Forth. (Unless they write most of
>the Forth compiler in Forth.) Maybe it would be useful for you to take
>some project you've already done and translate it into Forth. Not a
>literal translation, figure out how to use Forth tools to do it simpler.
I have both old projects and new one that may fit the bill. Pointers to
documentation more appropriate for the level of discussion will help
too.
>Maybe start by looking for things that get done more than once that you
>can factor out. Short definitions that become tools. Then later you
>might see ways to throw away whole sections and replace them with
>something simpler. And the point of this isn't just to improve your
>product (though making it simpler may remove some subtle bugs). It's to
>learn all the Forth tricks that you haven't thought to ask us about yet,
>that will make your main project much easier to create.
All well taken. One thing I'm figuring out is that reading descriptions
of how things work is more helpful than reading code that does it. For
example I have a pretty clear understading of CREATE DOES> because of
Brad Rodriguez's use of it in his "build an assembler in forth" article.
Thanks for taking the time to talk to me. I'll go work some more.
BAJ
none Byron Jeff
Brad Eckert <nospaa...@tinyboot.com> wrote:
>On Jun 27, 8:14 pm, byron@upstairs.(none) (Byron Jeff) wrote:
>>
>> Problem Definition: I want to develop an incremental word cross compiler
>> for the PIC. I hope this has been a "been there, done that." type
>> exercise for some of you. The idea is to develop and test the word on
>> the host, then cross compile the word into its PIC equivalent for
>> transport to the target. So for example say I develop a simple word on
>> the host:
>>
>> : add2 2 + ;
>
>Here are a couple of options:
>
>1. The compiler creates a word ADD2 on the host that pushes 2 onto the
>target's stack and tells the target to execute the "+" token. That
>tests the word on the host.
host or target? That's a real blended umbilical model there. It's an
interesting idea though.
>Another way to test on the host is to make
>the target model closely match the host model. For example, I
>currently run a 32-bit little-endian Forth on a 16-bit big-endian
>target just so my models match. In the case of the PIC, you could make
>the host more closely model the 16-bit target but it sounds to me like
>tackling that one would require more Forth experience on your part. My
>point about models is that you can think of Forth as a modeling
>language instead of a programming language and see where that takes
>you.
Good ideas.
>2. The compiler creates a data structure ADD2 on the host that
>contains information like execution address and it also compiles the
>definitions into tokens or whatever code the PIC expects to execute.
I was looking to do this after the fact. I guess I feel it's an
efficiency issue. Why compile for the target for 5 iterations if only
the last iteration is really going to the target?
>The host could download this code (ROM image) to the target right away
>or it could cache it until you actually test ADD2 or tell the host to
>flush the cache.
That's something along the lines of what I'm looking for.
>> 1. Instead of taking input from the normal input buffer, I want to
>> fundamentally parse a dictionary definition. The way I see it is
>> something like running "see <word>" taking the output and parsing it.
>> What I'm out of the loop with is accessing dictionary definitions
>> directly and parsing the words in those definitions.
>
>Once a definition is compiled, it's a black box. Your code doesn't
>have any business peeking inside.
Agreed. For a number of reasons the source should be captured as text at
the point of entry. Keeping the text of the definition facilitates both
compilation to multiple targets (host and target) and file storage.
>SEE is a decompiler (mostly a sanity
>checker), which happens to spit out Forth source in some
>implementations but assembly code in other implementations.
I've seen it. Capturing the source text is a better model.
>> 2. I know that create puts new words in the dictionary and allot will
>> allocate space. I'm unsure how to access that allotted space once I
>> created it or exactly how to get a pointer to the definition's alloted
>> area. Brad used C, to extend and write into the dictionary space for
>> example.
>
>The word >BODY is used to address data structures created by CREATE.
>In the case of target compilation, colon would create a data structure
>containing the word's execution address along with whatever other
>stuff you want. The text interpreter could use >BODY to get that
>address and execute it on the target, compile it into the ROM image,
>etc.
Thanks for that.
>As an example of a text interpreter, here is a snippet of a text
>interpreter that I load on top of an ANS Forth (including GForth),
>which I use to compile to a target.
[snippage]
I'll try to work my way through it.
>> 4. I know that Forth parses and does dictionary lookups. What kinds of
>> words should I be looking for to do these activities? Can parsing be
>> done from an arbitrary buffer?
>
>Parsing should be done from the input buffer, which could be the
>console or a file or a text string (see EVALUATE). These are all just
>streams of ASCII characters that are generally passed over once,
>although some fancy compilers may do more complex things.
Got it.
>I would recommend writing your tools in Forth to get experience
>writing Forth. It's a completely different skill from implementing
>Forths.
I'll working on finding projects to do in Forth in parallel with my
quest.
BAJ
none Byron Jeff
Jonah Thomas <j2th...@cavtel.net> wrote:
>byron@upstairs.(none) (Byron Jeff) wrote:
>> Jonah Thomas <j2th...@cavtel.net> wrote:
>> >byron@upstairs.(none) (Byron Jeff) wrote:
>> >> As we have discussed elsewhere in this thread, I'm only a novice to
>> >> the Forth language. For programming in general, embedded systems
>> >> development, and the like I know my way around the block.
>> >But if you want to use Forth, you need to learn how to use Forth.
>> >That doesn't need to take a long time, but it's a necessary step. So
>> >get some Forth compiler and practice with it.
>> I've been noodling with basic constructs. For me the only way to get a
>> feel for a language is to write a real application in it. This
>> compiler is the only task that's burning in my brain right. So I'm
>> going to jump into the deep end of the shark pool and take a swim.
>OK. It sounds like a good project. Maybe you can break it into little
>pieces and start with the ones you know how to do.
High level design is helpful. I've already learned one poor assumption
already just from the replies I've gotten.
>> The problem I'm having is that there's a pretty significant disconnect
>> between tutorial work and the tasks I want to do. I learned more about
>> what I wanted to pull off from Brad's Writing a Forth Assembler than
>> any of the three books I studied or the numerous guides/tutorials I've
>> read. There's a bit of a different skillset doing systems work as
>> opposed to application work.
>In Forth, pretty much everything you're talking about is advanced
>application work. Some of it is basic applicsation work.
I'm trying to get up to speed with advanced applications as quickly as I
can.
>> 1. Get the next word from the definition
>> 2. Look up the pic address for the word
>> 3. Generate a call to the pic address.
>> 4. Write the compiled call to the end of the pic compiled definition.
>> 5. Repeat 1-4 for each successive word to the end of the definition.
>
>OK. You want to have a variable that holds the first available code
>address on the PIC.
Yes.
>When you compile a command, you make a definition
>on the host that returns that address.
OK.
>You also make a second definition
>that puts PIC code on the host for the simulator to execute.
I'm not so sure about the simulator. The host is going to be tethered to
a real hardware target. That target will execute compiled forth words.
There's a debate as to whether or not a simulator needs to be in the
middle of this.
>Whenever you compile more code into the simulated definition,
Let's call it the target definition.
>you increment the
>variable to remind you about the PIC memory you're using up.
Agreed.
>So you can compile source code, and each word tells you the PIC address.
Bingo.
>Making the call to that address and writing the code to the end of the
>current definition ought to be easy.
I believe so.
>Your simulator has to use PIC addresses and convert them to real
>addresses to call, but that's easy.
Again not so sure about the simulator. I've always been thinking about a
test on host, compile and load to target, test on target model. A
simulator would either have to be an external application or would have
to be written in Forth. I'm really not trying to compound my problems
right now.
>> >It looks like the concession
>> >you're making to reality here is that you want to flash blocks at a
>> >time and do your final testing for whole blocks of words at once, in
>> >the hope that you'll only have to do it again a few times.
>>
>> Right. So test the word(s) on the host and do a limited number of
>> writes to the target. It's antithetical to the forth way because
>> generally forth machines operate from RAM.
>
>It looks to me like a slight complication.
Just an observation. Most of the tethered environments seem to target
seamless compilation and instant downloading along with only target
testing. RAM on the target facilitates all of these activities.
>> >> HOST
>> >> : add2 2 + ;
>> >> 5 add2 . ( do a bit of testing ) 7 ok
>> >> ( it works so let's get it to the target )
>> >>
>> >> TARGET
>> >> : add2 2 + ( why do I need to retype it? ) ;
>> >>
>> >> I already have the word I want in the host wordset. I don't want to
>> >> have to repeat typing it (and all of its error filled
>> >possibilities)> in the target wordset.
>> >
>> >One way is to put the word in a file, and load the file. And then you
>> >might as well keep your tests in a file too.
>>
>> Now that's a thought. You'd need some kind of versioning so that only
>> new and updated words are actually update.
>>
>> I'll go read up on file I/O. Thanks.
>
>You have HOST SIMULATE and TARGET . HOST compiles in Forth on the host,
>and you can use it to test your logic.
Yes.
>SIMULATE compiles PIC code on the host.
So does TARGET. No need to differentiate between the two.
>There's no particular reason to decompile the Forth code to
>reconstitute your source and recompile it as PIC unless you really want
>to.
I was initially thinking in a strictly interactive manner. Some
counsuling here has convinced me that the decompile/recompile route is
bad mojo. However I do want to ensure that the same source that's tested
on the host is the same source that's used to compile to the TARGET.
One way to ensure this and to facilitate longer term storage than
one session is to actually capture the text of the definition and save
it as part of the target environment. then it can be compiled for the
host, and the target, and saved to/reloaded from a file.
High level design changes before writing applications are a good thing.
>Probably simpler just to recompile the source.
Right. But you need the source to do it. So even if you just noodling
around you need to squirrel the source away as you type it.
>TARGET compiles on
>the PIC and if you designed your simulator so it runs exactly the same
>code that will run on the PIC then you might want to just dump that code
>to the PIC to put it in flash.
Like I said SIMULATE/TARGET are the same.
>> >' add2 picompile
>> >' add4 picompile
>>
>> Thanks for that too. The ' operates one word forward. Got it.
>
>That's an early-intermediate application-code thing.
Still trying to get up to speed.
>> >> 1. Words for accessing dictionary definitions like SEE does?
>> >
>> >Like SEE ? Ah. You get an xt, or anyway the address of a call, and
>> >you want to find the name. The newer Forth standards don't give you
>> >any way to do that since some Forth systems can't do it easily. The
>> >Forth-83 standard let you do that, I think. There was a word like
>> >>NAME . I don't remember whether you had to do >BODY >NAME or some
>> >other combination. You might check the documentation of your
>> >particular Forth, or look at the code with DUMP or something like
>> >that to see how it's actually laid out.
>>
>> Not standard. Got it. I'll figure out how gforth does it then.
>
>You could try SEE SEE . You might get something by SEE WORDS too.
Led me to some more words. This line is abandoned as a bad idea. I can
capture text. EVALUATE can parse text. So SEE/WORDS and the like can be
left by the wayside.
>Remember, your point in doing this is to decompile code so you can
>recompile it. If you re-use the source code you can skip this whole
>step.
Agreed. But I don't want the environment to devolve back to the
traditional EDIT/COMPILE/LOAD/EXECUTE model. If that were acceptable, I
could just use gforth/picforth model exactly the way it is now.
>> >> 3. words to address allotted memory in the dictionary?
>> >
>> >HERE 256 cells allot CONSTANT MY-DATA
>> >ok
>> >MY-DATA U.
>> >ok 21063284
>> >
>> >Give it the name to get the base address. Add whatever offset you
>> >want from there to get later addresses.
>> >
>> >CREATE MY-DATA2 32 CELLS ALLOT
>> >MY-DATA2 acts just like MY-DATA except the code gets compiled first.
>>
>> Excellent. and I can use ! and @ to access that memory.
>
>Yes, this is pretty basic application stuff.
Just double checking what I had read.
>> >When the one thing that's most getting in your way is that you don't
>> >know Forth, it looks like learning Forth would be the limiting factor
>> >that deserves some effort.
>>
>> I am certainly working on it. It's just that for the tasks that I need
>> to do, there isn't a generic book that describes it. I need the book
>> "Using Forth for Systems Programming."
>
>I didn't think about this stuff being advanced topics, but I guess maybe
>it is. So what you need is
>
>parsing words
>dictionary words
>compiling words
>data structures
>
>These are likely to do most of what you want.
I've found gforth's reference chapter organized with these types of
words. Will read over the next few days.
>Parsing words look ahead in the input buffer. PARSE WORD PARSE-WORD (or
>PARSE-NAME ) SKIP will let you build most of what you'd want, if you
>don't mind doing things one line at a time. You might find yourself
>re-inventing standard words that you happen not to know the names of.
>
>Dictionary words. FIND and SEARCH-WORDLIST find words. ' or ['] finds a
>word but it bombs out if the word isn't there. Only use that when you
>know for sure the word is defined. WORDLIST makes a new empty dictionary
>you can add to. SET-CURRENT and GET-CURRENT deal with the default
>dictionary. (Forthers call all the dictionaries together "the
>dictionary" and call individual ones vocabularies or wordlists.)
>GET-ORDER and SET-ORDER deal with which wordlists get searched by
>default and in which order. MARKER lets you get rid of the latest
>definitions.
>compiling words. IMMEDIATE makes a word execute during compilation so it
>will help the compiler do things instead of just get compiled itself.
>DOES> is worth special attention. RECURSE Probably some other words I
>DOES> haven't thought of would be extra useful.
>
>Data structures. CREATE DOES> (You probably don't want ALLOCATE ).
>HERE , C, ALLOT
>
>I think if you figure out how to use these they'll give you most of the
>tools you need to build whatever you want. Again, you might redesign
>some wheels that are universally available, but if it's easier to
>rewrite them than to find out about them then that's the way to go.
Excellent!!! I figured out how CREATE DOES> does it job. I'll read up on
the others after printing out this message.
Thanks so much for the help.
BAJ
none Byron Jeff
Saw it in passing. It's a traditional cross compiler written in python
instead of forth. Two problems for me is that it targets the 18F family,
which I haven't personally yet gotten to. The other is that the
traditional EDIT/COMPILE/DOWNLOAD/TEST cycle is exactly what I'm trying
to get away from.
If I get the model I desire, it should be possible to generate a
tethered, incremental compiling, interactive forth for the 18F simply by
generating the primitive kernel and burning the 18F part with it.
So I'll continue to work with the 16F family knowing that transisioning
isn't going to be too terribly difficult once I get to a certain point.
BAJ
Bruce McFarling
> Problem Definition: I want to develop an incremental word cross compiler
> for the PIC. I hope this has been a "been there, done that." type
> exercise for some of you. The idea is to develop and test the word on
> the host, then cross compile the word into its PIC equivalent for
> transport to the target.
One approach is to define a set of defining words ( : CREATE and
friends) that both compile the word in the host and cross-compile the
PIC version at the same time, and then "picupdate" to load the portion
of the PIC dictionary image that has not yet been installed.
... but you will still need the normal cross-compilation ability to
compile definitions specifically to the host or the target, because
there will be hardware features of the target that you will have to
provide dummies or emulating words for on the host.
Jonah Thomas
> Jonah Thomas <j2th...@cavtel.net> wrote:
> >byron@upstairs.(none) (Byron Jeff) wrote:
> >> Jonah Thomas <j2th...@cavtel.net> wrote:
> >> >byron@upstairs.(none) (Byron Jeff) wrote:
> >> 1. Get the next word from the definition
> >> 2. Look up the pic address for the word
> >> 3. Generate a call to the pic address.
> >> 4. Write the compiled call to the end of the pic compiled
> >definition.> 5. Repeat 1-4 for each successive word to the end of the
> >definition.
> >You also make a second definition
> >that puts PIC code on the host for the simulator to execute.
>
> I'm not so sure about the simulator. The host is going to be tethered
> to a real hardware target. That target will execute compiled forth
> words. There's a debate as to whether or not a simulator needs to be
> in the middle of this.
Great! Then it gets simpler.
So, you could CREATE a buffer the size of the chunk you want to flash at
one time.
And for each word you compile, you look up its PIC code and write the
compiled call into that buffer, and also you compile the Forth call into
the current Forth definition. When the buffer is full, every word except
the last one will already be compiled as PIC code.
So you can test all but the last word interactively before you compile
that last word -- ideally test each one right after you write it. Then
when you do fill the buffer, while the last word is incomplete, flash
the buffer, empty the buffer, and continue compiling PIC code. If
there's an error in the last word that went into the flashed part then
you'll have to save it again. But you get to test all the other words
and the last part of that one word interactively.
Or when there isn't enough room in your buffer for that last word, you
can flash what you have and waste just that much space. Maybe reclaim
the little scraps during one last flash when you're sure everything
works.
> Again not so sure about the simulator. I've always been thinking about
> a test on host, compile and load to target, test on target model. A
> simulator would either have to be an external application or would
> have to be written in Forth. I'm really not trying to compound my
> problems right now.
OK. There's a possible problem if the Forth on the host is different
from that on the PIC. Like, a 32-bit Forth on the host and a 16-bit
Forth on the target, it's vaguely possible you'll get something that
works on one but not the other. Tricky little bugs that don't happen
very often, maybe just often enough that you've forgotten about them
when they happen again. So there are some advantages to using a host
virtual machine that's as similar as possible to the PIC VM. On the
other hand, there are advantages to using whatever you're comfortable
with.
Bruce McFarling
> I'm not so sure about the simulator. The host is going to be tethered to
> a real hardware target. That target will execute compiled forth words.
> There's a debate as to whether or not a simulator needs to be in the
> middle of this.
Possibly not ... the target is the best simulator of itself you are
going to find, except when you need debugging support that the target
can't provide, and that is less common with incremental compilation.
> Just an observation. Most of the tethered environments seem to target
> seamless compilation and instant downloading along with only target
> testing. RAM on the target facilitates all of these activities.
> >> >> I already have the word I want in the host wordset. I don't want to
> >> >> have to repeat typing it (and all of its error filled
> >> >possibilities)> in the target wordset.
> >> >One way is to put the word in a file, and load the file. And then you
> >> >might as well keep your tests in a file too.
> >> Now that's a thought. You'd need some kind of versioning so that only
> >> new and updated words are actually update.
Or you could just copy and paste from the end of the history file, if
your host forth maintains an interpreter history file.
> >SIMULATE compiles PIC code on the host.
> So does TARGET. No need to differentiate between the two.
TARGET compiles PIC code to the PIC. If that involves compiling PIC
code to the host and doing a download and save at ";", its better if
that is built into target-oriented versions of ":" and ";" that run in
"TARGET" scope.
> Right. But you need the source to do it. So even if you just noodling
> around you need to squirrel the source away as you type it.
... which a history file does automatically
none Byron Jeff
none) (Byron Jeff <byron@upstairs.> wrote:
>Hi folks,
I'm back after doing some reading (thanks Elizabeth) and some coding.
I've been thinking about what features should core wordset should
exhibit and wanted to get some input on the matter.
In my original post I wrote:
>1. A specification for hosting Forth with a minimal kernel would be
>instructive for tool developers because it would abstract what one needs
>to implement Forth in a particular environment. One of my maxims to my
>students is "Make it work. Then make it pretty." In Forth's case I'd
>change that to "Make it work. Then make it fast." The concept of minimal
>kernels always gets shot down because of the resulting speed of the
>result. However, if a developer can get Forth working on the target,
>then optimizing words to make it faster can be done incrementally as you
>move forward. One crack at this was done in this thread:
>
>http://tinyurl.com/22c7sn
>
>with the specification here:
>
>http://www.quirkle.com/misc/forth.htm
>
>While I think Jim has the wrong target audience, the general Forth
>enthusiast, the idea has merit if targeted towards tool builders.
After some further thought I realize that there needs to be a balance
between minimalism and efficiency. So I've thought up a few rules of
thumb that may be helpful in developing core wordsets:
1. In every major grouping of words (arithmetic, logical, stack, etc.)
there is one word that can be used to derive most of the others in that
group. That word should be in the core wordset.
An example is in the logical group, all basic logic functions can be
derived from NAND. So NAND should be implemented in the core wordset.
2. If a word can be factored, consider putting the factors into the core
wordset. Continuing with NAND, it can be factored into AND and INVERT.
So a consideration should be made into implement the two more basic
words, though NAND can in fact implement both of the other two.
3. If the underlaying hardware implements a word's function natively,
consider recoding the word to use the native function. Three examples
from different groups (and their derivable word in parenthesis):
+ (-), BRANCH (?BRANCH), OR (NAND). While each of these words are
derivable from other words, they can be efficiently recoded to use the
native functionality.
4. Definitely consider recoding a word natively when there is a bunch of
complexity implementing with derived words. There is a fragment in a
c.l.f discussion named "What is Minimum Assembly Coded Words set in
Forth?" from 11 years ago where I believe Bernd Paysan shows how to
implement a right shift using only a NAND and =0. It literally tested
each bit in the original word and set the bit one bit position down if
the original bit was set. Painful but doable.
One question I have is if you have to implement multibyte operations by
hand (+,-, comparisons) is it better to use big endian or little endian
representation in memory? I found that ANS is neutral on the issue and
that either representation is compliant.
BAJ
Elizabeth D Rather
...
You're on the right track, but there's no point in restricting yourself
unnecessarily. No point in writing more words twice than is necessary;
some of these words are easier to do in code than high level anyway,
particularly since you say you're very familiar with PIC code.
Following is a rough list of the low-level code primitives we do in code:
* Structure words (run-time for DO, LOOP, IF, etc.)
* CREATE run-time
* 15 data stack primitives
* 9 R-stack primitives (including I & J)
* Logic: AND OR XOR INVERT
* 11 Arithmetic primitives (including some optimizations like 1+ and 2*
which have native instruction support)
* Misc: ALIGNED LSHIFT RSHIFT
* 12 comparison operators
* 5 memory access words (@ ! etc.)
* EXECUTE
On an 8051 these comprise about 1K (not counting heads, of course, since
we keep heads in the host). We've evolved this standard set over many
years, so doing the list for a new processor is pretty automatic.
There are some more code words in our systems, including support for the
multitasker, clock, I/O, etc., but I'm not including them as you're
asking about kernel stuff.
> One question I have is if you have to implement multibyte operations by
> hand (+,-, comparisons) is it better to use big endian or little endian
> representation in memory? I found that ANS is neutral on the issue and
> that either representation is compliant.
That really depends on the instruction set for the processor and the
"culture" using that processor (that is, what do most PIC programmers
do?). This is why ANS is neutral on the issue. Strive for whatever
makes the code cleanest.
Also, take care to allocate your registers in such a way as to make your
code simple and clean. Pick registers to use for S and R (stack
pointers), write a few primitives, and see if a different choice would
simplify.
Mikael Nordman
It is much better suited for hosting a forth VM for a incrementeal
programming style.
It features selfprogramming flash in 64 byte blocks, and the
program memory can easily be read via a pointer register.
It has three ram pointer registers and a readable and writable 16 bit
wide, 31 cells deep, HW return stack.
Also you get enough memory (ram 1.5 - 4 kbytes, flash 16 -128 kbytes,
eeprom 256 - 1024 bytes ).
The 18F also comes in convienient DIP packages with the nice PIC
peripherals.
I have also looked at the PIC24 series but it has a flash block erase
size of 1536 bytes which is not convinient for incremental writes. You
would need to waste 1536 bytes of ram for holding data during the flash
erase procedure.
Mikael
--
http://www.kolumbus.fi/oh2aun - FlashForth
none Byron Jeff
Mikael Nordman <poista...@turha.kolumbus.fi> wrote:
>I really recommend for you to switch to the PIC 18Fxxxx series.
I'll get there eventually. I literally have a double handful of 16F
parts and a virtually completely development system for them set up.
>It is much better suited for hosting a forth VM for a incrementeal
>programming style.
I think Elizabeth has convinced me that incremental programming is
orthogonal to other issues.
>It features selfprogramming flash in 64 byte blocks,
The 16F parts are the same.
>and the program memory can easily be read via a pointer register.
Now that has some utility in terms of coding literals. My plan in to
inline literal insertion instead of threading it as a call because since
I'm planning on doing a STC with the hardware PIC 16F stack (which isn't
addressible), I can't read the return address off the stack.
>It has three ram pointer registers and a readable and writable 16 bit
>wide, 31 cells deep, HW return stack.
All very cool features. Like I said eventually I'll get there.
>Also you get enough memory (ram 1.5 - 4 kbytes, flash 16 -128 kbytes,
>eeprom 256 - 1024 bytes ).
Enough is always a relative term. These are microcontrollers, so it's
very likely you're looking at single taking projects anyway.
The one thing that looks possible in the 18F arena is a completely self
hosted forth. With that much flash/ram it should be possible to actually
keep the dictionary and headers onboard.
>The 18F also comes in convienient DIP packages with the nice PIC
>peripherals.
Which is great for prototyping and hobby work. Another advantage that
the 18F family has is hardware USB and CAN interfaces on some parts. No
16F part carries either.
>I have also looked at the PIC24 series but it has a flash block erase
>size of 1536 bytes which is not convinient for incremental writes. You
>would need to waste 1536 bytes of ram for holding data during the flash
>erase procedure.
Ouch! One cheap trick is to attach a SPI/I2C serial FRAM and use it as a
holding area. FRAMS have virtually unlimited read/write life, can be
serially clocked up to 5 Mhz, is byte addressible, and when coupled with a
hardware SPI/I2C interfaces is pretty fast. So one could use it as a
holding area for flash blocks that are being updated and it'll only cost
you 3 I/O pins and possibly your syncronous serial interface.
Incremental developed Forth for the 16F family is a project I should
have tackled 10 years ago. But until recently I only had a dim awareness
of what Forth was all about. There are a ton of better chip families to
do embedded Forth development on. But I feel compelled to leave some
type of mark on PIC 16F Forth development before moving on into those
arenas.
BAJ
none Byron Jeff
Elizabeth,
I think (I'll know once I've done some implementation), that there's
some utility is getting a quick win when moving to a new system. We all
agree that a minimally coded kernel will have serious efficiency issues.
But that same minimally coded kernel facilitates getting a forth system
migrated quickly.
The higher level Forth words only needs to be written once. It then
instantly migrates. Consider PLUS and MINUS, which are both
implementable with the other core word:
: PLUS 0 SWAP - - ;
: MINUS INVERT 1 + + ;
Slower than a CODE implementation? Of course. But once it's written and
stuck in a file, you know they work.
So one can code the small subset, drop the rest of the high level kernel
in place and have a working, albeit slow, system. Then one recodes a
larger set incrementally, while mainting a working (but getting faster)
system as you go along.
You are correct that in this instance I know PIC code very well. But if
I were moving to a new chip architecture that I didn't know well, it
would be nice to only have a code a small subset to get migration going
and some testing on the target done.
>Following is a rough list of the low-level code primitives we do in code:
>
>* Structure words (run-time for DO, LOOP, IF, etc.)
>* CREATE run-time
>* 15 data stack primitives
>* 9 R-stack primitives (including I & J)
>* Logic: AND OR XOR INVERT
>* 11 Arithmetic primitives (including some optimizations like 1+ and 2*
>which have native instruction support)
>* Misc: ALIGNED LSHIFT RSHIFT
>* 12 comparison operators
>* 5 memory access words (@ ! etc.)
>* EXECUTE
Wow! That is quite a daunting set to tackle.
>On an 8051 these comprise about 1K (not counting heads, of course, since
>we keep heads in the host). We've evolved this standard set over many
>years, so doing the list for a new processor is pretty automatic.
And absolutely necessary for a professional outfit like Forth Inc. It
has a completely different view for one hobby guy who really just wants
to noodle with new architectures.
>There are some more code words in our systems, including support for the
>multitasker, clock, I/O, etc., but I'm not including them as you're
>asking about kernel stuff.
Cool.
>
>> One question I have is if you have to implement multibyte operations by
>> hand (+,-, comparisons) is it better to use big endian or little endian
>> representation in memory? I found that ANS is neutral on the issue and
>> that either representation is compliant.
>
>That really depends on the instruction set for the processor and the
>"culture" using that processor (that is, what do most PIC programmers
>do?).
The PIC is strictly a 8 bit machine. There is absolutely no standard in
terms of multibute numeric representations. IIRC I implemented by 16 bit
numbers as big endian primarily for readability in the simulator. Just
off the cuff little endian would be a win because both addition and
subtraction need to work on the LSB first.
>This is why ANS is neutral on the issue. Strive for whatever
>makes the code cleanest.
Will do.
>
>Also, take care to allocate your registers in such a way as to make your
>code simple and clean. Pick registers to use for S and R (stack
>pointers), write a few primitives, and see if a different choice would
>simplify.
The one optimization I plan from the jump is to keep the TOS in a
register. Since the PIC only has a single index register, it's important
to make the TOS (for the data stack) directly addressible.
Thanks,
BAJ
Mikael Nordman
> The one thing that looks possible in the 18F arena is a completely self
> hosted forth. With that much flash/ram it should be possible to actually
> keep the dictionary and headers onboard.
I spent last winter doing just that. The result is FlashForth.
You can find it at www.kolumbus.fi/oh2aun
> Incremental developed Forth for the 16F family is a project I should
> have tackled 10 years ago. But until recently I only had a dim awareness
> of what Forth was all about. There are a ton of better chip families to
> do embedded Forth development on. But I feel compelled to leave some
> type of mark on PIC 16F Forth development before moving on into those
> arenas.
>
Always nice to leave a mark, :-)
I was considering doing a self hosted PIC forth a few years back, but
really the PIC18F made it possible. There are other better chips, but
for hobbyists the PICs are quite optimal.
Mikael
Brad Eckert
>
> Ouch! One cheap trick is to attach a SPI/I2C serial FRAM and use it as a
> holding area. FRAMS have virtually unlimited read/write life, can be
> serially clocked up to 5 Mhz, is byte addressible, and when coupled with a
> hardware SPI/I2C interfaces is pretty fast. So one could use it as a
> holding area for flash blocks that are being updated and it'll only cost
> you 3 I/O pins and possibly your syncronous serial interface.
>
On a related note, you can run code out of a serial dataflash device
(like the AT26DF081A) via SPI at up to 70MHz. At that speed it
delivers 8M bytes/sec except when a branch is taken, at which time you
waste several slots loading the new address. I'm running one now at
50MHz. Having practically unlimited code space in a $1 part is kinda
cool. That might be worth investigating just because it's not tied to
any particular processor architecture for the VM implementation. When
you lose interest in PICs, there's always FPGAs, for example.
One word that's nice to have in a minimal kernel is [UNDEFINED]. You
use it to make known-good slow primitives (like the math operations of
eForth) when the CODE version of the word isn't defined. That lets you
port easily but not be very committed as to what is CODE and what is
high level.
Brad
none Byron Jeff
classes feel. I cover the obligatory chapter on memory management and
show how virutal memory using demand paging with cached multilevel page
tables represents one of the most efficient ways to map memory in a
modern OS. But that's a long long way from "load the program into a
contiguous memory space and execute it."
After some further reading of Brad's, Stephen's, and especially
Elizabeth's material, I believe I'm starting to grasp the nature of
metacompilation in Forth. The summary is that words are structured to
compile themselves and are organized into vocabularies that changes the
function of words depending on vocabulary search order.
Each word in the TARGET vocabulary needs 4 data elements in its definition:
1) target address of the word
2) possibly the length of the code
3) flags (such as INLINE and NOT TRANSFERRED (to target))
4) the actual code definition
this set would represent the parameter field(s) of the definition.
The code field would simply pull the target address of the word, create
a CALL instruction out of it (presuming not inline) and stick that
instruction at the end of the code definition currently being compiled.
It's starting to make more sense.
A question: is there a standard way to get an image of a vocabulary to
and from a file?
BAJ
none Byron Jeff
Brad Eckert <nospaa...@tinyboot.com> wrote:
>On Jul 13, 5:30 am, byron@upstairs.(none) (Byron Jeff) wrote:
>>
>> Ouch! One cheap trick is to attach a SPI/I2C serial FRAM and use it as a
>> holding area. FRAMS have virtually unlimited read/write life, can be
>> serially clocked up to 5 Mhz, is byte addressible, and when coupled with a
>> hardware SPI/I2C interfaces is pretty fast. So one could use it as a
>> holding area for flash blocks that are being updated and it'll only cost
>> you 3 I/O pins and possibly your syncronous serial interface.
>>
>
>On a related note, you can run code out of a serial dataflash device
>(like the AT26DF081A) via SPI at up to 70MHz. At that speed it
>delivers 8M bytes/sec except when a branch is taken, at which time you
>waste several slots loading the new address. I'm running one now at
>50MHz. Having practically unlimited code space in a $1 part is kinda
>cool.
I looked at those parts a long while ago when I was thinking about my
bytecode language. Good to know they are still around.
>One word that's nice to have in a minimal kernel is [UNDEFINED]. You
>use it to make known-good slow primitives (like the math operations of
>eForth) when the CODE version of the word isn't defined. That lets you
>port easily but not be very committed as to what is CODE and what is
>high level.
Is it the definition of the word here?
http://www.forth200x.org/defined.html
If so its usage isn't immediately apparent. Can you give an example of
its use when you get a chance? Thanks.
BAJ
Coos Haak
> A question: is there a standard way to get an image of a vocabulary to
> and from a file?
FYSFORTH (used long ago and developed at the Universty Utrecht) first for
Apple ][ and later PC (around 1986) had a system to write and read binary
absolute and relative relocatable images of portions of the code to disk.
I've rarely used the procedure, compiling from a text file was simple
enough.
I think it's too complicated for a small (tiny ;) PIC processor. The speed
of metacompilation on the PC/Mac is almost equal as normal compilation in
Forth. So why not INCLUDE a text file? It may be worth considering if you
are producing for the mass-market.
--
Coos
CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html
none Byron Jeff
Coos Haak <chf...@hccnet.nl> wrote:
>Op Sat, 14 Jul 2007 05:32:05 -0500 schreef none:
>
>> A question: is there a standard way to get an image of a vocabulary to
>> and from a file?
>
>FYSFORTH (used long ago and developed at the Universty Utrecht) first for
>Apple ][ and later PC (around 1986) had a system to write and read binary
>absolute and relative relocatable images of portions of the code to disk.
>I've rarely used the procedure, compiling from a text file was simple
>enough.
>
>I think it's too complicated for a small (tiny ;) PIC processor. The speed
>of metacompilation on the PC/Mac is almost equal as normal compilation in
>Forth. So why not INCLUDE a text file? It may be worth considering if you
>are producing for the mass-market.
Because one of my goals is building an incremental interactive
development system that does not have the edit/compile/download/execute loop.
The system I envision is an umbilical type with the code on the target
and the heads on the host. Even if I only retain word names and target
addresses across sessions, I don't want to have to leave forth to edit,
then reload a file with the edited header info each time I add
definitions. I can live with a single save/reload between sessions. But
I'd prefer not to have to do it during a session.
It's no biggie to iterate over the words in a vocabulary and dump/reload
the addresses/lengths/source associated with those words.
I'm not getting a clear sense of what modern forth development models
are. Do forthwrites typically develop interactively using the forth
interpreter or is it really a true edit with an external editor,
load/compile via include, then restest? I want to avoid the latter.
BAJ
Elizabeth D Rather
...
> Elizabeth's material, I believe I'm starting to grasp the nature of
> metacompilation in Forth. The summary is that words are structured to
> compile themselves and are organized into vocabularies that changes the
> function of words depending on vocabulary search order.
>
> Each word in the TARGET vocabulary needs 4 data elements in its definition:
>
> 1) target address of the word
> 2) possibly the length of the code
Haven't found this necessary, really. If you're downloading a newly
compiled definition, you have its starting address and the address of
the next unused location, and the difference is the size of your new
code. If you're building an image for later download, you track the
image's size.
> 3) flags (such as INLINE and NOT TRANSFERRED (to target))
Well, some definitions may be inline-able, but that's a property of the
INTERPRETER part, not the target version. And we keep a global switch
that governs whether we're transferring definitions as they're compiled
vs. building an image for download, so no need to track individual
definitions.
> 4) the actual code definition
>
> this set would represent the parameter field(s) of the definition.
>
> The code field would simply pull the target address of the word, create
> a CALL instruction out of it (presuming not inline) and stick that
> instruction at the end of the code definition currently being compiled.
>
> It's starting to make more sense.
Good.
> A question: is there a standard way to get an image of a vocabulary to
> and from a file?
No. In the first place, a vocabulary is a logical entity whose
components are not in any particular physical location. Traditionally,
Forth is compiled from source to executable form. You may wish to save
an image of your kernel or app for flashing at the start of a
development session, but not individual vocabularies.
Elizabeth D Rather
...
> are. Do forthwrites typically develop interactively using the forth
> interpreter or is it really a true edit with an external editor,
> load/compile via include, then restest? I want to avoid the latter.
It sounds as though you're trying to do away with source in order to
further your interactive development model. IMO that's throwing the
baby out with the bathwater. Source is useful, because it's able to
capture information that cannot realistically be kept in your executable
object (even the host version). Most critically, source contains
*comments* that explain what your code is doing, why it's doing it that
way, how to use various definitions, and (most important) stack comments
for each definition.
A Forth development model is really a hybrid: you do write code using
either a Forth-resident or external editor. It's really easier to think
through your logic when you're working with a related group of
definitions in a file. But that doesn't lock you in to testing huge
code batches.
The first step is to develop a working kernel that will include
necessary primitives and support for your XTL. That pretty much needs
to be a downloadable or flashable image generated from a (short) list of
files.
Subsequently, you can write definitions in your editor and test them
individually by doing a copy/paste into the development window of your
Forth, whereupon they're compiled & downloaded to your target for
testing. This saves re-typing and preserves your source along with its
vital comments.
Incidentally, the time to write your comments is when you're writing the
code, not after the fact. You write what this definition will do and
how it's intended to be used, then its name, and its stack comment.
This is then in front of you as you work out the actual code. If you
realize that something is awkward (e.g. you need your stack arguments in
a different order) you immediately change the comment. This way you
have much less to juggle in your head. Newbies often find it helpful to
have stack comments interspersed in the definition, not just at the
beginning.
Assuming you successfully test a bunch of definitions in a session, you
then add that file to the list of those that generate your kernel for
the next session. Thus, you always start a session with a kernel that
includes your tested code thus far, and can go on from there.
Does this help?
Elizabeth D Rather
Elizabeth D Rather wrote:
...
>
> Assuming you successfully test a bunch of definitions in a session, you
> then add that file to the list of those that generate your kernel for
> the next session. Thus, you always start a session with a kernel that
> includes your tested code thus far, and can go on from there.
Unless you're superhuman, things will go wrong in testing. It's nice to
be able to 'reset' to the point where you have a clean system with just
your downloaded kernel (or kernel plus tested code). Since you
obviously don't want to download it again, keep a checksum in, say, the
last few bytes so you can check that the downloaded image matches the
kernel image you last compiled. If they don't match, you have to do a
download, because that means the heads in your host won't match the
locations in your target.
none Byron Jeff
>...
>> I'm not getting a clear sense of what modern forth development models
>> are. Do forthwrites typically develop interactively using the forth
>> interpreter or is it really a true edit with an external editor,
>> load/compile via include, then restest? I want to avoid the latter.
>
>It sounds as though you're trying to do away with source in order to
>further your interactive development model.
That's not exactly it. I really want to integrate the source into the
interactive development model.
>IMO that's throwing the
>baby out with the bathwater. Source is useful, because it's able to
>capture information that cannot realistically be kept in your executable
>object (even the host version). Most critically, source contains
>*comments* that explain what your code is doing, why it's doing it that
>way, how to use various definitions, and (most important) stack comments
>for each definition.
No doubt. I'm just not understanding why after all of the years of forth
development, that an integrated editor for source never emerged? Forth
clearly started as an interactive system. While it retains some of those
interactive elements, the prevailing mindset seems to be usage as a
compiler with the source edited offline.
>A Forth development model is really a hybrid: you do write code using
>either a Forth-resident or external editor.
Forth resident editor seems to be very system specific. Here's what
gforth's manual had to say on the subject of editing forth source:
http://www.delorie.com/gnu/docs/gforth/gforth_59.html
---------------------
If you have tried out the examples in this section, you will probably
have typed them in by hand; when you leave Gforth, your definitions will
be lost. You can avoid this by using a text editor to enter Forth source
code into a file, and then loading code from the file using include (see
section 5.16.1 Forth source files). A Forth source file is processed by
the text interpreter, just as though you had typed it in by hand(7).
Gforth also supports the traditional Forth alternative to using text
files for program entry (see section 5.17 Blocks).
---------------------
While I completely understand why external editors are useful (I'm a vi
guy myself) and completely agree with the freedom that it provides
different users, it would still be useful to be able to function within
the Forth environment proper.
GForth is quite helpful in a couple of respects here. First it does
integrate the readline line editing capability into its command line.
Second it keeps a history file of previous commands entered. The two
facilities are quite close to being enough to a working integrated
editing system.
>It's really easier to think
>through your logic when you're working with a related group of
>definitions in a file. But that doesn't lock you in to testing huge
>code batches.
No real argument with this. Just wondering why it has to be an external
editor.
You've spent a lot of time explaning to me the transparancy aspect of
Forth cross compilers. I'm just trying to figure out why that
transparancy doesn't extend to editing systems.
>The first step is to develop a working kernel that will include
>necessary primitives and support for your XTL.
I've made progress here implementing and testing a few primitives
(pushnum, add, drop). I realized early on that keeping the TOS in a
register isn't a win on the PIC. I'm implementing the stack using the
PIC's only indirect register pair.
>That pretty much needs
>to be a downloadable or flashable image generated from a (short) list of
>files.
While I see the utility of writing a Forth based assembler for the PIC,
I'm simply using my existing assembler to write the XTL.
>Subsequently, you can write definitions in your editor and test them
>individually by doing a copy/paste into the development window of your
>Forth, whereupon they're compiled & downloaded to your target for
>testing.
>This saves re-typing and preserves your source along with its
>vital comments.
But why isn't there an integrated source editor generally available?
Especially something that's smart enough to know what definitions have
actually been changed and need to be recompiled?
I know it's broad, but I'm curious as to why development gravitated to
the separate editor along with the Forth interpreter.
I got from my reading that older self hosted Forths implemented file
blocks and that file block editors were integrated into the self hosted
system. Once the migration to implement Forths on top of existing OS
started, why didn't a native integrated file editor emerge?
>Incidentally, the time to write your comments is when you're writing the
>code, not after the fact.
Absolutely. In my assembly code I generally put a header comment on
every routine and comment any code that isn't self documented.
>You write what this definition will do and
>how it's intended to be used, then its name, and its stack comment.
>This is then in front of you as you work out the actual code. If you
>realize that something is awkward (e.g. you need your stack arguments in
>a different order) you immediately change the comment. This way you
>have much less to juggle in your head. Newbies often find it helpful to
>have stack comments interspersed in the definition, not just at the
>beginning.
All good ideas.
I guess the reason that I'm harping a bit on development style and
integrated editing is the fact that if I keep all of my level
definitions in a file and recompile each time I make a change, I'm going
to have to track revisions that need to be downloaded to the target
anyway. It's important to me because I'm not operating on a RAM based
target system where rewrites are effortless. Flash is slow to write and
has limited write cycles (only 1000 writes/cell guaranteed across the
temp range for the PIC though the typical expectancy at 25C is 100,000
writes).
If the definitions are in an external file, then loading that file means
recompiling everything. Now that in itself isn't too problematic. But
tracking actual definition changes is.
It would be easier to track such changes when editing the source. But
with the typical development model of editing externally, there's no
transparent way I can see to track version definitions short of
compiling the definition and then hashing the result, while keeping the
current hash value of that definition on flash elsewhere. Any
definitions that are missing or have a different hash value would need
to be rewritten to flash.
I was thinking I could integrate such version tracking in a resident
Forth editor for the source. When I change a line, only that line needs
to be recompiled and version tracked. In short it's a visual view of the
existing Forth command line system.
As for keeping comments, that was one reason that I wanted to store
target definitions in string format. The other is to have the definition
available for compilation to both the target and the host. I realize the
error of my initial thought of recompiling the host version of a
definition directly to the target. But having a string version of the
source and compiling for both is a possibility.
>Assuming you successfully test a bunch of definitions in a session, you
>then add that file to the list of those that generate your kernel for
>the next session. Thus, you always start a session with a kernel that
>includes your tested code thus far, and can go on from there.
But that's right back to batch development. That means you've thown out
my baby (small incremental development flashing only changes) with the
bathwater of using the external editor in a standard
edit/compile/download/execute system.
There is a hybrid model between the two extremes of executable code only
and externally filed code only. Source stored as text in forth (read and
written to files between sessions of course) with incremental
compilation and download is that hybrid.
I'm trying to figure out how to edit natively in Forth in that hybrid.
>Does this help?
Some. Thanks for the input.
BAJ
Elizabeth D Rather
> In article <139i7v6...@news.supernews.com>,
> Elizabeth D Rather <erath...@forth.com> wrote:
>> IMO that's throwing the
>> baby out with the bathwater. Source is useful, because it's able to
>> capture information that cannot realistically be kept in your executable
>> object (even the host version). Most critically, source contains
>> *comments* that explain what your code is doing, why it's doing it that
>> way, how to use various definitions, and (most important) stack comments
>> for each definition.
>
> No doubt. I'm just not understanding why after all of the years of forth
> development, that an integrated editor for source never emerged? Forth
> clearly started as an interactive system. While it retains some of those
> interactive elements, the prevailing mindset seems to be usage as a
> compiler with the source edited offline.
We started out that way: for 10 years Forth was standalone, with *only*
an integrated editor. Editors started out very simple, and added
features over time. In the 80's, as more co-resident Forths came to be,
there was increasing demand from programmers to be able to use the
editors they already knew and were comfortable with. In particular,
there was a lot of customer resistance to block editors.
Frankly, companies that devote themselves (or whole divisions of
themselves) to making really great editors can do a better job than
Forth developers whose primary interest is making a great *Forth* and
using it to develop applications.
...
> While I completely understand why external editors are useful (I'm a vi
> guy myself) and completely agree with the freedom that it provides
> different users, it would still be useful to be able to function within
> the Forth environment proper.
>
> GForth is quite helpful in a couple of respects here. First it does
> integrate the readline line editing capability into its command line.
> Second it keeps a history file of previous commands entered. The two
> facilities are quite close to being enough to a working integrated
> editing system.
So do all our products. But I find that what I type in the command
window is quite different from what I would want in a well-constructed
source file.
>> It's really easier to think
>> through your logic when you're working with a related group of
>> definitions in a file. But that doesn't lock you in to testing huge
>> code batches.
>
> No real argument with this. Just wondering why it has to be an external
> editor.
Because, as I said above, it's probably a better editor. Back in the
70's and 80's an insane amount of effort went into developing and
maintaining Forth editors. Having used commercial editors for over 10
years now, I don't find that the editing/testing cycle is any less
intimate. And our customers are a lot happier.
The most thoroughly integrated Forth editor is the one in Power
MacForth. Its users swear by it, but some years ago I tried really hard
to learn to use it (as did several of our engineers) and it just wasn't
comfortable.
> You've spent a lot of time explaning to me the transparancy aspect of
> Forth cross compilers. I'm just trying to figure out why that
> transparancy doesn't extend to editing systems.
...
>
>> Subsequently, you can write definitions in your editor and test them
>> individually by doing a copy/paste into the development window of your
>> Forth, whereupon they're compiled & downloaded to your target for
>> testing.
>
>> This saves re-typing and preserves your source along with its
>> vital comments.
>
> But why isn't there an integrated source editor generally available?
> Especially something that's smart enough to know what definitions have
> actually been changed and need to be recompiled?
Most programmers' editors do this kind of tracking for you, along with a
lot of other features.
Our products are closely integrated with most commercial editors (about
20, actually) such that from the command window if you type LOCATE FOO
it will get the source for FOO along with some number of lines above and
below (so you can see the context) from the editor, and if you type EDIT
FOO it switches context to your editor with FOO's file opened and your
cursor placed at the start of FOO's definition. Actually, you can
select a word and use a right-mouse-click to get these functions, too.
You can copy/paste in both directions. I'm really not sure what more
would be useful.
> I know it's broad, but I'm curious as to why development gravitated to
> the separate editor along with the Forth interpreter.
>
> I got from my reading that older self hosted Forths implemented file
> blocks and that file block editors were integrated into the self hosted
> system. Once the migration to implement Forths on top of existing OS
> started, why didn't a native integrated file editor emerge?
Actually, there are still a few. The ones I've used have been (IMO)
inferior to commercial editors, and offered no conveniences I couldn't
get with the kind of integration I described above.
...
> I guess the reason that I'm harping a bit on development style and
> integrated editing is the fact that if I keep all of my level
> definitions in a file and recompile each time I make a change, I'm going
> to have to track revisions that need to be downloaded to the target
> anyway. It's important to me because I'm not operating on a RAM based
> target system where rewrites are effortless. Flash is slow to write and
> has limited write cycles (only 1000 writes/cell guaranteed across the
> temp range for the PIC though the typical expectancy at 25C is 100,000
> writes).
That's why I suggested working with individual definitions via
copy/paste from your editor into the Forth command window. And I think
that's the correct direction, since when you're typing in the command
window you're less likely to worry about comments and nice formatting.
You can write a coherent chunk of your project in a file using the
editor of your choice, and test individual definitions in Forth by
pasting one into the command window, exercising it, and moving on to the
next.
> If the definitions are in an external file, then loading that file means
> recompiling everything. Now that in itself isn't too problematic. But
> tracking actual definition changes is.
But you don't need to load the whole file every time. You do bottom-up
testing, working with each definition (copy/paste) until you get it
right then move on to the next.
> It would be easier to track such changes when editing the source. But
> with the typical development model of editing externally, there's no
> transparent way I can see to track version definitions short of
> compiling the definition and then hashing the result, while keeping the
> current hash value of that definition on flash elsewhere. Any
> definitions that are missing or have a different hash value would need
> to be rewritten to flash.
I've never done any of that nor felt the need to. We do use
version-control systems on our products and larger projects, which is
helpful. And another good reason to use external editors, because they
integrate with version-control systems better than internal Forth editors.
What you need to control via checksum or hash is your kernel. But
nothing should go into that until it's been thoroughly tested and is
therefore pretty stable.
> I was thinking I could integrate such version tracking in a resident
> Forth editor for the source. When I change a line, only that line needs
> to be recompiled and version tracked. In short it's a visual view of the
> existing Forth command line system.
A *line*? Forth doesn't know anything about lines. And you surely
aren't going to be able to change a definition in the middle of your
program. It just doesn't work that way. What you *can* do is
repeatedly compile the most recent definition (or few). When you test,
you always get the most recent version.
> As for keeping comments, that was one reason that I wanted to store
> target definitions in string format. The other is to have the definition
> available for compilation to both the target and the host. I realize the
> error of my initial thought of recompiling the host version of a
> definition directly to the target. But having a string version of the
> source and compiling for both is a possibility.
Yep. It's called a source file.
>> Assuming you successfully test a bunch of definitions in a session, you
>> then add that file to the list of those that generate your kernel for
>> the next session. Thus, you always start a session with a kernel that
>> includes your tested code thus far, and can go on from there.
>
> But that's right back to batch development. That means you've thown out
> my baby (small incremental development flashing only changes) with the
> bathwater of using the external editor in a standard
> edit/compile/download/execute system.
Only if you assume you repeatedly download this image. If you only
migrate things into the kernel that are well-tested, you rarely have to
re-flash the whole thing.
> There is a hybrid model between the two extremes of executable code only
> and externally filed code only. Source stored as text in forth (read and
> written to files between sessions of course) with incremental
> compilation and download is that hybrid.
>
> I'm trying to figure out how to edit natively in Forth in that hybrid.
This is a topic that's really difficult to capture in a discussion like
this. If you would spend five minutes sitting at one of our
cross-compilers you would understand what it's taking weeks to thrash
through this way. It's like trying to teach swimming by email to
someone who refuses to get wet until the course is over.
John Doty
> ...
> window is quite different from what I would want in a well-constructed
> source file.
That reflects a limitation in the Forth dialect you choose to use:
commands are interpreted pidgin, while programs are in a full-featured
language. Your choice. But we've known for three decades that compiling
commands makes the full language available at both levels, at no cost.
--
John Doty, Noqsi Aerospace, Ltd.
http://www.noqsi.com/
--
Specialization is for robots.
Elizabeth D Rather
> Elizabeth D Rather wrote:
>> ...
>> I find that what I type in the command window is quite different from
>> what I would want in a well-constructed source file.
>
> That reflects a limitation in the Forth dialect you choose to use:
> commands are interpreted pidgin, while programs are in a full-featured
> language. Your choice. But we've known for three decades that compiling
> commands makes the full language available at both levels, at no cost.
I realize that you find it hard to resist a chance to make some snide
remarks, but you're completely misunderstanding me.
When I'm typing in the command window I'm usually testing hypotheses,
hardware, etc. I'll use word names like FOO or TRY rather than
carefully chosen names, because these definitions will never become part
of my program in their present form. I pay little attention to line
breaks or formatting. When I'm working on a source file I assume I'm
writing for posterity, and take appropriate care. The issue you're
referring to (interpreting vs compiling) is completely irrelevant.
none Byron Jeff
Elizabeth D Rather <erath...@forth.com> wrote:
>none Byron Jeff wrote:
>> In article <139i7v6...@news.supernews.com>,
>> Elizabeth D Rather <erath...@forth.com> wrote:
>...
>> development, that an integrated editor for source never emerged?
>We started out that way: for 10 years Forth was standalone, with *only*
>an integrated editor. Editors started out very simple, and added
>features over time. In the 80's, as more co-resident Forths came to be,
>there was increasing demand from programmers to be able to use the
>editors they already knew and were comfortable with. In particular,
>there was a lot of customer resistance to block editors.
I want to note the comfort point. I'll come back to it later.
>Frankly, companies that devote themselves (or whole divisions of
>themselves) to making really great editors can do a better job than
>Forth developers...
>using it to develop applications.
>
>> GForth is quite helpful in a couple of respects here. First it does
>> integrate the readline line editing capability into its command line.
>> Second it keeps a history file of previous commands entered. The two
>> facilities are quite close to being enough to a working integrated
>> editing system.
>
>So do all our products. But I find that what I type in the command
>window is quite different from what I would want in a well-constructed
>source file.
So editing and testing are two completely different activities in your
head. I'm more fluid. They can be separate or integrated to me.
Remember the comfort point above.
>>> It's really easier to think
>>> through your logic when you're working with a related group of
>>> definitions in a file. But that doesn't lock you in to testing huge
>>> code batches.
>>
>> No real argument with this. Just wondering why it has to be an external
>> editor.
>
>Because, as I said above, it's probably a better editor. Back in the
>70's and 80's an insane amount of effort went into developing and
>maintaining Forth editors. Having used commercial editors for over 10
>years now, I don't find that the editing/testing cycle is any less
>intimate. And our customers are a lot happier.
>
But that precludes even having one, just one, integrated editor?
Co-resident computers are completely different than they were in the
1980's. I find the virtually complete abandonment of integration a bit
disconcerting. It's funny because usually I argue from the other side
complaning when systems force the use of an integrated system and
disallows (or greatly restricts) external components.
>The most thoroughly integrated Forth editor is the one in Power
>MacForth. Its users swear by it, but some years ago I tried really hard
>to learn to use it (as did several of our engineers) and it just wasn't
>comfortable.
There's that comfort point again.
So let's cut to the chase...
>> There is a hybrid model between the two extremes of executable code only
>> and externally filed code only. Source stored as text in forth (read and
>> written to files between sessions of course) with incremental
>> compilation and download is that hybrid.
>>
>> I'm trying to figure out how to edit natively in Forth in that hybrid.
>
>This is a topic that's really difficult to capture in a discussion like
>this. If you would spend five minutes sitting at one of our
>cross-compilers you would understand what it's taking weeks to thrash
>through this way. It's like trying to teach swimming by email to
>someone who refuses to get wet until the course is over.
Now on that comfort point. I'm a Linux guy. You don't have a product for
my platform. I've spent over 20 years getting a comfort level with my
systems environment. The 5 minutes you're asking for would require a
complete overhaul of that environment and my carefully cultivated
mindset.
It's the same argument for PICs. I started with them in the 16F54 era of
the early 1990s. For embedded systems projects they serve the need I
need them to serve. From a price, availablity, and prototypeability (is
that a word?) standpoint they serve my needs quite well.
You've spoken of comfort in your post. I'm very comfortable in the
environments that I've built for myself. I'm trying to find ways to
integrate Forth into those environments instead of turning my
environments on their heads simply to fit the perceived existing Forth
model.
The cool thing about it is that Forth is designed with extensibility in
mind. Nothing that I've asked about so far is impossible, or even too
terribly difficult, to pull off. It just that most of it is outside of
the prevailing mainstream forth mindset.
No problem. I'm very comfortable trying to swim in sand. ;-)
BAJ
Stephen Pelc
wrote:
>No doubt. I'm just not understanding why after all of the years of forth
>development, that an integrated editor for source never emerged? Forth
>clearly started as an interactive system. While it retains some of those
>interactive elements, the prevailing mindset seems to be usage as a
>compiler with the source edited offline.
Like other vendors, we spent a lot of time writing block/screen
editors. When we switched to regular text files, we didn't have
to reinvent the editor wheel. Good editors are dirt cheap. We
could then spend our hard-earned money on improving the Forth
system.
Engineering is a compromise and perfection never arrives.
Stephen
--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
Anton Ertl
>I'm just not understanding why after all of the years of forth
>development, that an integrated editor for source never emerged?
Maybe you are thinking of something like Holon, which has recently had
a new release.
>But why isn't there an integrated source editor generally available?
>Especially something that's smart enough to know what definitions have
>actually been changed and need to be recompiled?
On desktop and server machines incremental recompilation would not
save a significant amount of time. Also, the way that compilation
normally works in Forth (old calls still call the old version of the
word), just recompiling the changed words is not enough, you also need
to recompile all their callers, etc.
But IIRC Holon does something like you seem to be wanting.
There's also the forth.el Emacs mode that came with TILE (and that's
the ancestor of gforth.el), that ran the Forth system (e.g., TILE or
Gforth) as a process in an Emacs shell-mode window; you can edit a
file, and then send a part of the file that you just changed to the
Forth process, which see's it as user input and interprets/compiles
it. I never used this, so AFAIK this was not maintained in gforth.el
and probably no longer works there.
>I guess the reason that I'm harping a bit on development style and
>integrated editing is the fact that if I keep all of my level
>definitions in a file and recompile each time I make a change, I'm going
>to have to track revisions that need to be downloaded to the target
>anyway. It's important to me because I'm not operating on a RAM based
>target system where rewrites are effortless. Flash is slow to write and
>has limited write cycles (only 1000 writes/cell guaranteed across the
>temp range for the PIC though the typical expectancy at 25C is 100,000
>writes).
>
>If the definitions are in an external file, then loading that file means
>recompiling everything. Now that in itself isn't too problematic. But
>tracking actual definition changes is.
Preserving flash has not been a problem for most Forth systems, so few
systems address this problem. I guess the easiest solution for your
problem is to cool the PIC in your development system nicely, then it
should survive long enough (i.e., until it's outdated) even with full
recompilation every time. The production systems are not going to see
that many recompilations, so no special cooling is needed for them.
However, a cross-development system that wanted to minimize flash
writes could still look like a recompile-everything system to the
user. It would need to keep track of what's currently in flash, and
how to get to a state corresponding to the current source from the
current flash state with minimal writes. However, given the write
blocking that flash usually requires this could be a tough problem
(i.e., patching a byte in an existing definition causes a rewrite of a
whole block).
>It would be easier to track such changes when editing the source. But
>with the typical development model of editing externally, there's no
>transparent way I can see to track version definitions short of
>compiling the definition and then hashing the result, while keeping the
>current hash value of that definition on flash elsewhere. Any
>definitions that are missing or have a different hash value would need
>to be rewritten to flash.
Yes, something like that. But you also have to redirect calls to
these definitions. BTW, why keep hashes, just compare the new version
of the code to the existing one.
>I was thinking I could integrate such version tracking in a resident
>Forth editor for the source. When I change a line, only that line needs
>to be recompiled and version tracked.
Lots of my editing reorganizes code or changes comments or
documentation without changing the functionality of the definition.
Comparing after compilation has is better than relying on knowledge
which line changed.
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2007: http://www.complang.tuwien.ac.at/anton/euroforth2007/
John Doty
> John Doty wrote:
>> Elizabeth D Rather wrote:
>>> ...
>>> I find that what I type in the command window is quite different from
>>> what I would want in a well-constructed source file.
>>
>> That reflects a limitation in the Forth dialect you choose to use:
>> commands are interpreted pidgin, while programs are in a full-featured
>> language. Your choice. But we've known for three decades that
>> compiling commands makes the full language available at both levels,
>> at no cost.
>
> I realize that you find it hard to resist a chance to make some snide
> remarks, but you're completely misunderstanding me.
Not really. I understand where you're coming from. I have been there.
But you're unwilling to broaden your horizons enough to understand where
*I'm* coming from. You have never tried a Forth dialect like LSE, and I
doubt that you would be willing to understand it anyway.
>
> When I'm typing in the command window I'm usually testing hypotheses,
> hardware, etc. I'll use word names like FOO or TRY rather than
> carefully chosen names,
When testing, why choose a name at all?
> because these definitions will never become part
> of my program in their present form.
While in LSE, you get it right, then slap a name and a colon on it. In
many cases, it's "gee, I've typed that three times in the last five
minutes, it must deserve to be a definition".
> I pay little attention to line
> breaks or formatting.
The need for careful formatting in traditional Forth stems from a design
that inhibits factoring. One may avoid locals, but you still have to use
block structure for loops and conditionals.
You could argue that this is independent of the interpretation issue,
but it really isn't. In a compile-only Forth you want to design the
language with commands in mind. One premeditates commands much less than
programs: block structured commands are difficult to compose. But Goeke
simplified flow control in LSE, making it more "concatenative", and
therefore not only easier to use, but a better match to Forth's linguistics.
> When I'm working on a source file I assume I'm
> writing for posterity, and take appropriate care.
I used to work that way in LSE. I added block structure to make it more
like traditional Forth. Took care to be clear from a "structured
programming" viewpoint. Wound up being less clear than the folks who
just slapped names on their commands. An illuminating experiment.
> The issue you're
> referring to (interpreting vs compiling) is completely irrelevant.
Only if you keep your eyes tightly shut to anything that might
contradict your worldview.
Elizabeth D Rather
> Elizabeth D Rather wrote:
...
>> remarks, but you're completely misunderstanding me.
>
> Not really. I understand where you're coming from. I have been there.
> But you're unwilling to broaden your horizons enough to understand where
> *I'm* coming from. You have never tried a Forth dialect like LSE, and I
> doubt that you would be willing to understand it anyway.
I'll try anything that solves a problem that I have or offers an
advantage that looks useful. So far, from your descriptions of LSE and
code examples I've seen no advantages and many disadvantages.
>> When I'm typing in the command window I'm usually testing hypotheses,
>> hardware, etc. I'll use word names like FOO or TRY rather than
>> carefully chosen names,
>
> When testing, why choose a name at all?
So I can easily exercise the word repeatedly, for example with varying
parameters.
>> because these definitions will never become part of my program in
>> their present form.
>
> While in LSE, you get it right, then slap a name and a colon on it. In
> many cases, it's "gee, I've typed that three times in the last five
> minutes, it must deserve to be a definition".
Well, since I never have to retype it at all, that isn't an issue.
>> I pay little attention to line breaks or formatting.
>
> The need for careful formatting in traditional Forth stems from a design
> that inhibits factoring. One may avoid locals, but you still have to use
> block structure for loops and conditionals.
>
> You could argue that this is independent of the interpretation issue,
> but it really isn't. In a compile-only Forth you want to design the
> language with commands in mind. One premeditates commands much less than
> programs: block structured commands are difficult to compose. But Goeke
> simplified flow control in LSE, making it more "concatenative", and
> therefore not only easier to use, but a better match to Forth's
> linguistics.
The reason for good formatting is to facilitate reading, whether there
are sturctures or not.
>> When I'm working on a source file I assume I'm writing for posterity,
>> and take appropriate care.
>
> I used to work that way in LSE. I added block structure to make it more
> like traditional Forth. Took care to be clear from a "structured
> programming" viewpoint. Wound up being less clear than the folks who
> just slapped names on their commands. An illuminating experiment.
Well, I can't comment on your programming style any more knowledgeably
than you can on mine. The difference is I don't try.
John Doty
> John Doty wrote:
>> Elizabeth D Rather wrote:
>>> I realize that you find it hard to resist a chance to make some snide
>>> remarks, but you're completely misunderstanding me.
>>
>> Not really. I understand where you're coming from. I have been there.
>> But you're unwilling to broaden your horizons enough to understand
>> where *I'm* coming from. You have never tried a Forth dialect like
>> LSE, and I doubt that you would be willing to understand it anyway.
>
> I'll try anything that solves a problem that I have or offers an
> advantage that looks useful. So far, from your descriptions of LSE and
> code examples I've seen no advantages and many disadvantages.
Filtered through your prejudices and lack of experience, I'm not
surprised at this judgment. Yes, lack of experience.
>
>>> When I'm typing in the command window I'm usually testing hypotheses,
>>> hardware, etc. I'll use word names like FOO or TRY rather than
>>> carefully chosen names,
>>
>> When testing, why choose a name at all?
>
> So I can easily exercise the word repeatedly, for example with varying
> parameters.
Sure. But before that stage, why bother? You really don't understand
what I'm talking about, do you?
>
>>> because these definitions will never become part of my program in
>>> their present form.
>>
>> While in LSE, you get it right, then slap a name and a colon on it. In
>> many cases, it's "gee, I've typed that three times in the last five
>> minutes, it must deserve to be a definition".
>
> Well, since I never have to retype it at all, that isn't an issue.
Most commands are throwaways. No point in naming and preserving them
all! Besides, who retypes? Triple click, paste. Nearly all code is
one-liners the way I work.
Of course that's just one approach. I often work top-down, which
requires more premeditation. I still keep most definitions small and
simple. LSE facilitates this and in some cases requires this. The word
is the only capsule for code.
>
>>> I pay little attention to line breaks or formatting.
>>
>> The need for careful formatting in traditional Forth stems from a
>> design that inhibits factoring. One may avoid locals, but you still
>> have to use block structure for loops and conditionals.
>>
>> You could argue that this is independent of the interpretation issue,
>> but it really isn't. In a compile-only Forth you want to design the
>> language with commands in mind. One premeditates commands much less
>> than programs: block structured commands are difficult to compose. But
>> Goeke simplified flow control in LSE, making it more "concatenative",
>> and therefore not only easier to use, but a better match to Forth's
>> linguistics.
>
> The reason for good formatting is to facilitate reading, whether there
> are sturctures or not.
Yes. But the best formatting is simply to keep definitions short.
Algolish structures get in the way, and don't really fit the Forth
paradigm well anyway.
>
>>> When I'm working on a source file I assume I'm writing for
>>> posterity, and take appropriate care.
>>
>> I used to work that way in LSE. I added block structure to make it
>> more like traditional Forth. Took care to be clear from a "structured
>> programming" viewpoint. Wound up being less clear than the folks who
>> just slapped names on their commands. An illuminating experiment.
>
> Well, I can't comment on your programming style any more knowledgeably
> than you can on mine.
I understand your style. I understand its history and how the nature of
the dialect you use influences it. But your inexperience in programming
(essentially a one-language programmer, yikes!) gives you no way to see
these things.
If you were a musician who had spent a lifetime playing and listening to
Mozart, never any other composer, you wouldn't even really understand
Mozart's music, although you'd no doubt be very good at playing it.
> The difference is I don't try.
Yes. That is the origin of your lack of experience. That is why you
cannot even comment knowledgeably about Forth dialects off the
traditional Forth path. You really don't understand Forth: you have no
perspective from which to do so.
The difference is I *do* try. Ever since Jon Sachs handed me a stack of
papers by you and Chuck and asked "What do you think of this?" Over 30
years ago. An interesting and innovative programming technique, then. I
wish it had evolved and gotten stronger rather than being frozen in a
primitive state and pretty much dying out. But I'm still getting some
mileage out of it. It helps that Jon Sachs and Bob Goeke really did
bring in new perspectives, yielding improved Forths. I wish Bob had
published...
Elizabeth D Rather
> Elizabeth D Rather wrote:
>> John Doty wrote:
>>> Elizabeth D Rather wrote:
...
>>>> hypotheses, hardware, etc. I'll use word names like FOO or TRY
>>>> rather than carefully chosen names,
>>>
>>> When testing, why choose a name at all?
>>
>> So I can easily exercise the word repeatedly, for example with varying
>> parameters.
>
> Sure. But before that stage, why bother? You really don't understand
> what I'm talking about, do you?
No. And it's pretty clear you don't understand what I'm talking about,
either.
>>>> because these definitions will never become part of my program in
>>>> their present form.
>>>
>>> While in LSE, you get it right, then slap a name and a colon on it.
>>> In many cases, it's "gee, I've typed that three times in the last
>>> five minutes, it must deserve to be a definition".
>>
>> Well, since I never have to retype it at all, that isn't an issue.
>
> Most commands are throwaways. No point in naming and preserving them
> all! Besides, who retypes? Triple click, paste. Nearly all code is
> one-liners the way I work.
You were the one that complained of typing something three times, not me.
...
>>> I used to work that way in LSE. I added block structure to make it
>>> more like traditional Forth. Took care to be clear from a "structured
>>> programming" viewpoint. Wound up being less clear than the folks who
>>> just slapped names on their commands. An illuminating experiment.
>>
>> Well, I can't comment on your programming style any more knowledgeably
>> than you can on mine.
>
> I understand your style. I understand its history and how the nature of
> the dialect you use influences it. But your inexperience in programming
> (essentially a one-language programmer, yikes!) gives you no way to see
> these things.
What you write doesn't show any understanding of my style or experience
at all. You have never seen me work, you have never seen my code. You
don't even know anything about my professional background, which
includes quite a lot beyond Forth. It is your style in this newsgroup
to throw out ignorant, misinformed assertions such as those above,
unsupported by any facts or evidence.
For you to accuse me of blindness or prejudice is a classic case of the
pot calling the kettle black.
John Passaniti
> Filtered through your prejudices and lack of experience, I'm not
> surprised at this judgment. Yes, lack of experience.
Ironic. I felt the same about you when I described a utilitarian
application of reflection that you classed as "clever" in the negative
sense of the word. Divorced from the details, divorced from a language
that provides an effective interface to such metadata, and divorced from
my actual real-world experience, you offered a statement based more on
prejudice than engineering sense.
And that would be fine-- if you admitted that. But you don't. And I'm
curious what breach of logic and fairness allows you to pretend you're
above it all.
> If you were a musician who had spent a lifetime playing and listening to
> Mozart, never any other composer, you wouldn't even really understand
> Mozart's music, although you'd no doubt be very good at playing it.
Interesting metaphor. How doesn't the same apply to you?
Here's what I find fascinating: You seem to be saying that Elizabeth
can't appreciate the wonder that is LSE because she hasn't *experienced*
it. Oh, she can read about the programming model, the syntax, and how
it is implemented-- all that good stuff-- but she can't truly appreciate
LSE without *experiencing* it. Did I get that right?
Yet in a conversation a while back-- where you fearlessly stated that it
wasn't possible to implement LSE in (ANS) Forth-- you admitted to not
programming in (ANS) Forth yourself. Or put in other words, you haven't
*experienced* it.
So Elizabeth's lack of experience with every variation on a Forth theme
makes her unable to comment authoritatively when she says LSE has
advantages and disadvantages. Yet your same lack of experience with the
mother language of LSE makes you... apparently an expert who we should
all respect.
> Yes. That is the origin of your lack of experience. That is why you
> cannot even comment knowledgeably about Forth dialects off the
> traditional Forth path. You really don't understand Forth: you have no
> perspective from which to do so.
Reread what you wrote to Elizabeth and explain to me why you're not a
hypocrite, considering your own stated lack of experience with (ANS) Forth.
John Doty
> What you write doesn't show any understanding of my style or experience
> at all.
Then explain it better. While you're at it, explain how it is that you
believe "many" astronomers still use Forth, when I know it must be very few.
> You have never seen me work, you have never seen my code.
I've read your papers. Those don't contain your code?
> You
> don't even know anything about my professional background, which
> includes quite a lot beyond Forth.
I know you repeatedly assert your ignorance of programming beyond Forth.
Indeed, I have stated I find this hard to credit. So which is it,
willful ignorance or misrepresentation?
If somebody came to you and offered a contract to recast work you'd
already done by other means into silicon VLSI hardware (something I
guess you've never done), would you reject the offer or start studying?
You can probably guess what I did...
> It is your style in this newsgroup
> to throw out ignorant, misinformed assertions such as those above,
> unsupported by any facts or evidence.
How many astronomers actually are using Forth these days? You've
asserted "many". I know this is major misrepresentation. The last time I
checked a sample of thousands, I couldn't find any.
You constantly assert your ignorance of programming. "I don't know much
about C", etc., etc. And that's been your excuse for not understanding
what I'm saying.
Jon Sachs learned a lot from you. And he passed it along, with
improvements, to the rest of us at MIT at the time. But you seem to have
learned nothing back, although it *was* offered. Three decades later,
you're still stuck in the 70's. I can't be, that won't work in my world
any more.
John Doty
> John Doty wrote:
>> Filtered through your prejudices and lack of experience, I'm not
>> surprised at this judgment. Yes, lack of experience.
>
> Ironic. I felt the same about you when I described a utilitarian
> application of reflection that you classed as "clever" in the negative
> sense of the word. Divorced from the details, divorced from a language
> that provides an effective interface to such metadata, and divorced from
> my actual real-world experience, you offered a statement based more on
> prejudice than engineering sense.
Your opinion.
>
> And that would be fine-- if you admitted that. But you don't. And I'm
> curious what breach of logic and fairness allows you to pretend you're
> above it all.
I'm not above it all. I'm in the trenches. Muddy down here. Let's work
on drainage, OK?
>
>> If you were a musician who had spent a lifetime playing and listening
>> to Mozart, never any other composer, you wouldn't even really
>> understand Mozart's music, although you'd no doubt be very good at
>> playing it.
>
> Interesting metaphor. How doesn't the same apply to you?
It simply doesn't. I've made a living as a systems programmer, an
applications programmer, a production engineer, an electrical engineer,
a systems engineer, and a scientist. For the ASCA space mission, I was
both a scientific investigator and the project engineer for the Silicon
Imaging Spectrometer, a feat of "switch hitting" that is unheard of in
any other space mission as far as I know. I've written significant code
in FORTRAN, PL/I, BASIC, Algol-60, PL/M, C, Python, LSE, Scheme,
Mathematica, ... . Also a large collection of assembly languages. I
wrote the structured programming tools Bob Frankston used to write
VisiCalc, the first spreadsheet
(http://www.frankston.com/public/?name=implementingVisicalc). I worked
on one of the first internationalized word processors in the late 70's.
I've done astronomy from submillimeter to gamma rays, and from the
Solar system to the most distant exploding stars, an enormous range of
physical processes and observational technique. I've designed
electronics ranging from super efficient power converters to measurement
chains with 2 electrons RMS noise. At MIT, the physics theorist next
door used me as a mathematics consultant. I've also built spark chambers
for high energy physics, worked on code and instrumentation for
thermonuclear fusion research, and worked on imaging for military
surveillance. The last memo I sent to one of my customers was about
mechanical engineering issues. Last year the bulk of my business was
spacecraft operations. This year it's VLSI design and systems
engineering. Next year, who knows? Whatever, it'll be fun!
No, it doesn't apply. I'm not a specialist. My professors taught me that
I could learn to do anything if I worked at it. I took that lesson
seriously.
>
> Here's what I find fascinating: You seem to be saying that Elizabeth
> can't appreciate the wonder that is LSE because she hasn't *experienced*
> it. Oh, she can read about the programming model, the syntax, and how
> it is implemented-- all that good stuff-- but she can't truly appreciate
> LSE without *experiencing* it. Did I get that right?
A broad experience base is essential to judgment. She has repeatedly
denied that she has such a base.
>
> Yet in a conversation a while back-- where you fearlessly stated that it
> wasn't possible to implement LSE in (ANS) Forth-- you admitted to not
> programming in (ANS) Forth yourself. Or put in other words, you haven't
> *experienced* it.
Not quite. I've noodled with it (as with many other techniques) and
found it wanting. I would change my mind if a significant community
would adopt it. A handful of enthusiasts, as bright as they are (why do
you suppose I keep coming back here?) don't, in the end, count. But
Python has my attention, definitely, although it isn't really my style
(but I never let that stop me). And ColorForth has my attention too: not
very practical, but maybe a stepping stone to the future. But ANS Forth
is essentially the same old Forth Jon Sachs used as a stepping stone 30
years ago. Been there, gone beyond it. But not far enough yet ...
Elizabeth D Rather
> Elizabeth D Rather wrote:
>
>> What you write doesn't show any understanding of my style or
>> experience at all.
>
> Then explain it better.
That's not the topic here.
> While you're at it, explain how it is that you
> believe "many" astronomers still use Forth, when I know it must be very
> few.
When all else fails you trot out this red herring? If I had any
reliable numbers (and there are no reliable numbers for any language
used in any field like this that I know of) I'd call them 'many' and
you'd call the same number 'few'.
>> You have never seen me work, you have never seen my code.
>
> I've read your papers. Those don't contain your code?
If you've read them you shouldn't be asking ;-) But code samples in
papers on Forth don't necessarily represent project code or one's style
when working on projects.
>> You don't even know anything about my professional background, which
>> includes quite a lot beyond Forth.
>
> I know you repeatedly assert your ignorance of programming beyond Forth.
> Indeed, I have stated I find this hard to credit. So which is it,
> willful ignorance or misrepresentation?
I have said that I haven't programmed in C. That's true. But I spent
over 10 years programming in Fortran (and some variants), several
versions of Algol, COBOL, APL, and about a dozen assembly languages.
I've read K&R on C and taken courses in Snobol and Smalltalk. And I've
managed several C projects, with some excellent C programmers.
For the last 15 years or so I've been mostly in company and project
management. Projects have been primarily (though not exclusively) in
Forth. Most of the programmers who've worked for me have rich and
varied backgrounds, and I learn a lot from them, from our customers (who
have used a lot of different languages, and many do not use Forth
exclusively), and from the students in my courses.
During the period of the ANSI process I and other TC members conducted
an intensive study of various Forth implementations and variants.
> If somebody came to you and offered a contract to recast work you'd
> already done by other means into silicon VLSI hardware (something I
> guess you've never done), would you reject the offer or start studying?
I'd accept the offer and hire an expert in the field to work with me.
> You can probably guess what I did...
Plunged in like a well-meaning amateur?
>> It is your style in this newsgroup to throw out ignorant, misinformed
>> assertions such as those above, unsupported by any facts or evidence.
>
> How many astronomers actually are using Forth these days? You've
> asserted "many". I know this is major misrepresentation. The last time I
> checked a sample of thousands, I couldn't find any.
Changing the subject doesn't help.
...
>
> Jon Sachs learned a lot from you. And he passed it along, with
> improvements, to the rest of us at MIT at the time. But you seem to have
> learned nothing back, although it *was* offered. Three decades later,
> you're still stuck in the 70's. I can't be, that won't work in my world
> any more.
As nearly as I can tell from what you write here, your view of Forth is
stuck in the 70's and Sach's reaction to it.
John Doty
> John Doty wrote:
>> Elizabeth D Rather wrote:
>>
>>> What you write doesn't show any understanding of my style or
>>> experience at all.
>>
>> Then explain it better.
>
> That's not the topic here.
>
>> While you're at it, explain how it is that you believe "many"
>> astronomers still use Forth, when I know it must be very few.
>
> When all else fails you trot out this red herring? If I had any
> reliable numbers (and there are no reliable numbers for any language
> used in any field like this that I know of) I'd call them 'many' and
> you'd call the same number 'few'.
And I'd be right. The number is so close to zero that they are very hard
to find. Astronomers using C, C++, Fortran, Java, Python, Perl,
Mathematica, Matlab, IDL, and even Visual Basic are much more numerous.
>
>>> You have never seen me work, you have never seen my code.
>>
>> I've read your papers. Those don't contain your code?
>
> If you've read them you shouldn't be asking ;-) But code samples in
> papers on Forth don't necessarily represent project code or one's style
> when working on projects.
>
>>> You don't even know anything about my professional background, which
>>> includes quite a lot beyond Forth.
>>
>> I know you repeatedly assert your ignorance of programming beyond
>> Forth. Indeed, I have stated I find this hard to credit. So which is
>> it, willful ignorance or misrepresentation?
>
> I have said that I haven't programmed in C. That's true. But I spent
> over 10 years programming in Fortran (and some variants), several
> versions of Algol, COBOL, APL, and about a dozen assembly languages.
> I've read K&R on C and taken courses in Snobol and Smalltalk. And I've
> managed several C projects, with some excellent C programmers.
Then you have less programming experience than I have. Substitute
Smalltalk's descendant Objective-C and the only one above I haven't
*used* is COBOL. I've used a bunch more. And the languages above are all
more than 30 years old. Things have changed a lot. You should try
Python. Or to really bend your mind, Mathematica.
>
> For the last 15 years or so I've been mostly in company and project
> management. Projects have been primarily (though not exclusively) in
> Forth. Most of the programmers who've worked for me have rich and
> varied backgrounds, and I learn a lot from them, from our customers (who
> have used a lot of different languages, and many do not use Forth
> exclusively), and from the students in my courses.
>
> During the period of the ANSI process I and other TC members conducted
> an intensive study of various Forth implementations and variants.
Yet you didn't "get" STOIC.
>
>> If somebody came to you and offered a contract to recast work you'd
>> already done by other means into silicon VLSI hardware (something I
>> guess you've never done), would you reject the offer or start studying?
>
> I'd accept the offer and hire an expert in the field to work with me.
Then you'd probably fail at a project where knowledge of the application
is more important than knowledge of the tool.
>
>> You can probably guess what I did...
>
> Plunged in like a well-meaning amateur?
Studied. Got advice from experts. Looked at the problem from different
angles. Plunged in. Succeeded. State of the art performance for 1/10 the
power and 1/100 the mass. Now doing final simulations for the second
generation chip. VLSI is taking over my business ;-)
>
>>> It is your style in this newsgroup to throw out ignorant,
>>> misinformed assertions such as those above, unsupported by any facts
>>> or evidence.
>>
>> How many astronomers actually are using Forth these days? You've
>> asserted "many". I know this is major misrepresentation. The last time
>> I checked a sample of thousands, I couldn't find any.
>
> Changing the subject doesn't help.
It's part of your pattern of misrepresentation.
> ....
>>
>> Jon Sachs learned a lot from you. And he passed it along, with
>> improvements, to the rest of us at MIT at the time. But you seem to
>> have learned nothing back, although it *was* offered. Three decades
>> later, you're still stuck in the 70's. I can't be, that won't work in
>> my world any more.
>
> As nearly as I can tell from what you write here, your view of Forth is
> stuck in the 70's and Sach's reaction to it.
Nah, 80's. Heyday of the LSE dialect. But the difference is I'm not in
denial about the decline of Forth. Looking for ideas to recover a tool
that's lost for most practical purposes. Lost, but not replaced...
But who's changing the subject now? I repeat, Jon learned a lot from
you, but you learned nothing from him, despite the fact that it was
offered and there was plenty to learn. Outside your box...
yao Pymy user
> John Doty wrote:
[...]
> As nearly as I can tell from what you write here, your view of Forth is
> stuck in the 70's and Sach's reaction to it.
Anyway (I'm not convinced ;-) and FYI
Wolfgang Hagen:
"The style of sources:
Remarks on the theory and history of programming languages"
<http://www.whagen.de/get.php?page=publications/2005/DigitalMediaRoutledge/RT2241_C0091.pdf>
on german:
"Der Stil der Sourcen
Anmerkungen zur Theorie und Geschichte der Programmiersprachen"
<http://www.whagen.de/publications/STILSOUR/STILSOUR.HTM>
John Doty
> Elizabeth D Rather wrote:
>
>> John Doty wrote:
> [...]
>> As nearly as I can tell from what you write here, your view of Forth is
>> stuck in the 70's and Sach's reaction to it.
I didn't write that, Elizabeth did!
Alex McDonald
The grammar's excellent, and the spelling is spot on, but the rest is
pure Jeff Fox.
--
Regards
Alex McDonald
John Doty
> The grammar's excellent, and the spelling is spot on, but the rest is
> pure Jeff Fox.
>
Never met the man. And if you'd check history of this group, you'd find
I've often disagreed with him. However, he *is* one of the interesting
characters that keep me coming back: at least he's thinking about the
future, not denying the sorry state of the present. Also not polluting
the group with endless RfD discussion that never gets anywhere.
Alex McDonald
> Alex McDonald wrote:
>
>> The grammar's excellent, and the spelling is spot on, but the rest is
>> pure Jeff Fox.
>>
>
> Never met the man. And if you'd check history of this group, you'd find
> I've often disagreed with him. However, he *is* one of the interesting
> characters that keep me coming back: at least he's thinking about the
> future, not denying the sorry state of the present.
But *all* you seem to do is harp on the sorry state of the present;
that, and indulging yourself in pub bragging contests about how many
languages you can order a beer in while scribbling out designs for
crinkle cut chips on napkins.
> Also not polluting
> the group with endless RfD discussion that never gets anywhere.
>
One of the beauties of the RfD process for me is the challenges it
throws up. How do I write UTF-8 applications for a UTF-16 OS? Why do
commercial vendors place so much emphasis on backwards compatability?
You find them irritating, I find them illuminating.
On the other hand, your continual carping I find polluting. At the end
of the day I realise its about being discriminating in one's own terms
in what to read and what to reply to. As long as your happy posting the
stuff you do, then I suppose I'll just have to suffer your endless
repetitions that never get anywhere, your personal and snide
observations, and your lack of substance; and you'll have to suffer the
endless RfDs in return.
Or, of course, you could stop reading them. You add RfD to your
killfile; I'm adding Doty to mine.
--
Regards
Alex McDonald
John Passaniti
>> Ironic. I felt the same about you when I described a utilitarian
>> application of reflection that you classed as "clever" in the negative
>> sense of the word. Divorced from the details, divorced from a
>> language that provides an effective interface to such metadata, and
>> divorced from my actual real-world experience, you offered a statement
>> based more on prejudice than engineering sense.
>
> Your opinion.
Yes, but it's an *informed* opinion based on real-world experience and
regarding a specific, actual application. Returning to the example,
when you offered your pronouncement against reflection as a "clever"
technique, it was purely abstract. No details. No discussion. You
didn't even try to engage me in a conversation where you would have
learned more about the application in question to try to poke holes in
the design-- or offer something simpler. Nope, just generic blanket
statements. And even if I'm being generous and think your statements
come from actual experience, you don't even detail the specific
experience you have to prove your point.
So sorry, but an informed opinion based on reality does win over a
take-it-on-faith religious argument every time. Engineering is funny
like that.
You, like a handful of other people here, suffer from the same thing.
You assume reputations and history and who you know lends credibility to
your (unsupported) statements. So let me see if I get this straight--
when you comment on a particular technique (let's use reflection again
as an example) without a shred of justification for your views, we're
just supposed to accept your statements... because you've done an
assortment of interesting things.
Where I come from (and where I work), people don't coast on reputations
and history and who they know. When I make a statement, it is assumed
(and sometimes tested) that I can justify that statement. And if I
can't, an appropriate answer is not to pull out my resume and start
quoting my (unrelated) career successes as you have here.
I'm guessing you wanted to show us your broad range of experience.
Terrific. You being an expert in any of the areas you cited doesn't
necessarily lend authority or credibility on *other* areas. Yet you
seem to keep coming back to your /argumentum ad verecundiam/ fallacy.
John Doty
> John Doty wrote:
>> Alex McDonald wrote:
>>
>>> The grammar's excellent, and the spelling is spot on, but the rest is
>>> pure Jeff Fox.
>>>
>>
>> Never met the man. And if you'd check history of this group, you'd
>> find I've often disagreed with him. However, he *is* one of the
>> interesting characters that keep me coming back: at least he's
>> thinking about the future, not denying the sorry state of the present.
>
> But *all* you seem to do is harp on the sorry state of the present;
> that, and indulging yourself in pub bragging contests about how many
> languages you can order a beer in while scribbling out designs for
> crinkle cut chips on napkins.
I've offered a Forth I think might be a stepping stone to the future.
I've talked about possible future directions. I've invited people to
share what they think might be a viable future (but don't base it on the
failure of Forth 94, please).
Elizabeth's constant oversell of her expertise is irritating to me for
sure. Vast areas of willful ignorance combined with overspecialization
are a recipe for disastrously poor judgment. I see way too much of this
in the space program: sometimes it gets people killed. But more commonly
it leads to stagnation.
But there are others here with real breadth.
>
>> Also not polluting the group with endless RfD discussion that never
>> gets anywhere.
>>
>
> One of the beauties of the RfD process for me is the challenges it
> throws up. How do I write UTF-8 applications for a UTF-16 OS? Why do
> commercial vendors place so much emphasis on backwards compatability?
Well, of course we know why that is: that's their business. And I have
no problem with that: let 'em have Forth 94. But I need a Forth for the
21st century. Especially one that scientists and engineers can return
to. And that's an area where I disagree with Jeff: he wants a Forth for
elite programmers. Ugh.
> You find them irritating, I find them illuminating.
All they show is how diabolical Forth 94 is: it can't be fixed. It's a
stable fixed point in the design space. Like a sunken ship.
>
> On the other hand, your continual carping I find polluting. At the end
> of the day I realise its about being discriminating in one's own terms
> in what to read and what to reply to.
How about telling me what future you think Forth has. Forget Forth 94
and tell me how you'd design a Forth that could attract users with real
problems to solve, not just specialists and enthusiasts.
> As long as your happy posting the
> stuff you do, then I suppose I'll just have to suffer your endless
> repetitions that never get anywhere, your personal and snide
> observations, and your lack of substance; and you'll have to suffer the
> endless RfDs in return.
>
> Or, of course, you could stop reading them. You add RfD to your
> killfile; I'm adding Doty to mine.
Your privilege.
John Doty
> John Doty wrote:
>>> Ironic. I felt the same about you when I described a utilitarian
>>> application of reflection that you classed as "clever" in the
>>> negative sense of the word. Divorced from the details, divorced from
>>> a language that provides an effective interface to such metadata, and
>>> divorced from my actual real-world experience, you offered a
>>> statement based more on prejudice than engineering sense.
>>
>> Your opinion.
>
> Yes, but it's an *informed* opinion based on real-world experience and
> regarding a specific, actual application.
Could an EE without extensive CS training have understood your code?
> Returning to the example,
> when you offered your pronouncement against reflection as a "clever"
> technique, it was purely abstract.
The problem is that it's unfamiliar to many. So you create a situation
where only a specialist can understand the code. Bad news.
> No details. No discussion. You
> didn't even try to engage me in a conversation where you would have
> learned more about the application in question to try to poke holes in
> the design-- or offer something simpler.
Ok. Show me.
> Nope, just generic blanket
> statements. And even if I'm being generous and think your statements
> come from actual experience, you don't even detail the specific
> experience you have to prove your point.
But I have. I've talked about some of the times I've hung myself by
being too clever. Like turning the original LSE into something like
traditional Forth. Big mistake. The old klunky code that the non-experts
wrote without my extensions was much more maintainable. Heck, the klunky
stuff is now more readable to me than the stuff I wrote back then with
the extensions. Now, that's a real experiment with a real outcome (and
certainly not what I wanted!). Do you have anything like that?
Or my more recent mistake of using currying and other functional methods
in the preliminary analysis of my chip architecture. Functional
programming is lots of fun, but explaining it to people without a
background in it is trouble. In the end, it wasn't worth it. Should have
just used klunky, straightforward code.
If these aren't real world experiences, I don't know what might satisfy you.
>
> So sorry, but an informed opinion based on reality does win over a
> take-it-on-faith religious argument every time. Engineering is funny
> like that.
Elizabeth is constantly asking us to take her judgments on faith. Why
don't you attack her?
>
> You, like a handful of other people here, suffer from the same thing.
> You assume reputations and history and who you know lends credibility to
> your (unsupported) statements.
Gee, you misrepresent my background, so I set the record straight. Then,
somehow, I'm banking on my reputation. But those who work with me know
that's exactly how I don't work. The hardware doesn't care about your
reputation, that's for sure. One of the really great things about the
space mission team I built at MIT was that they were a bunch of
skeptics: I could count on them to catch my every mistake (and, of
course, not just mine. Although they seemed to take great pleasure when
it was me... ). We'd have heated arguments, but at the end of the day we
were friends. The right answer would emerge from that dialectic.
> So let me see if I get this straight--
> when you comment on a particular technique (let's use reflection again
> as an example) without a shred of justification for your views, we're
> just supposed to accept your statements... because you've done an
> assortment of interesting things.
No, you should think about it. Does reflection clarify code to the
majority of technical people who don't understand what it is? Is it
important that the logic of your code be accessible to a reader with a
different background from yours?
>
> Where I come from (and where I work), people don't coast on reputations
> and history and who they know. When I make a statement, it is assumed
> (and sometimes tested) that I can justify that statement.
Frankly, my customers don't check on me enough. And I tell them that.
Makes me nervous. Part of why I'm spending 10^15 CPU cycles on
simulations this week. I've had the "final" design of the new chip for
almost a week now, but it's #$%@ hard to be sure it's really right. It
would be better, though, if I wasn't the one making up the simulation
cases because they are surely influenced by my prejudices of what might
go wrong. The real problem will hit me from an unexpected direction...
> And if I
> can't, an appropriate answer is not to pull out my resume and start
> quoting my (unrelated) career successes as you have here.
Everything is related. The world is very complicated, but when you have
the breadth to see the connections it gets a little simpler.
>
> I'm guessing you wanted to show us your broad range of experience.
> Terrific. You being an expert in any of the areas you cited doesn't
> necessarily lend authority or credibility on *other* areas. Yet you
> seem to keep coming back to your /argumentum ad verecundiam/ fallacy.
I will claim that broad experience and a willingness to study something
new every day is a better foundation for good judgment than narrow
specialization and willful ignorance. Do you disagree? Doesn't make for
infallibility though. Wish it did ;-)
yao Pymy user
> yao Pymy user wrote:
>> Elizabeth D Rather wrote:
>>
>>> John Doty wrote:
>> [...]
>>> As nearly as I can tell from what you write here, your view of
>>> Forth is stuck in the 70's and Sach's reaction to it.
> I didn't write that, Elizabeth did!
Sure and I've quoted correctly.
But your contributions here to Forth, programming languages and
against the priesthood reminds me to post those links to an
article with remarks to the history of programming languages.
--
"Forth is the last bastion of DIY", Charles Moore
Brad Eckert
Is that kind of like not "getting" CATCH/THROW in the thread
http://groups.google.com/group/comp.lang.forth/msg/b228b688cc3a3182?hl=en&
?
Brad
John Doty
I have no idea why that would make you think I don't "get" CATCH/THROW.
Maybe an analogy would help: I own two chainsaws. I also own a bowsaw. I
tend to get the bowsaw out more than the chainsaws. Especially if I have
a helper without chainsaw experience.
Brad Eckert
> Brad Eckert wrote:
> > On Jul 18, 6:04 am, John Doty <j...@whispertel.LoseTheH.net> wrote:
> >> Yet you didn't "get" STOIC.
>
> > Is that kind of like not "getting" CATCH/THROW in the thread
>
> I have no idea why that would make you think I don't "get" CATCH/THROW.
>
> Maybe an analogy would help: I own two chainsaws. I also own a bowsaw. I
> tend to get the bowsaw out more than the chainsaws. Especially if I have
> a helper without chainsaw experience.
>
Different application domains. Elizabeth likes CATCH/THROW because she
has some complex pieces of code that can experience I/O or other
errors in multiple places. But you have simpler error handling
requirements so you use error flags. Both you and Elizabeth understand
that it's easy to abuse CATCH/THROW, I think. You're both right but
you still bump heads. "Fascinating", as Spock would say.
But back to the OT, building a new Forth, there is a question of which
Forth to build. Yes, Forth usage is in decline. But it will never go
away because it's too good at what it does. Forth is words and stacks,
as Chuck says. ANS Forth adds wordlists and restrictions to promote C-
style portability.
As I see it, the evolution of Forth isn't the problem. The evolution
of reusable code (like libraries and applications) is the problem. A
body of code with environmental dependencies will tend to encourage
the Forth vendors to support those dependencies. For example, the FSL
assumes a separate floating point stack so all the Forths support
that. Much of my code deals with bytes and wouldn't work if chars
weren't bytes, so I wantonly embrace that dependency and make sure all
my targets support bytes. If a lot of code did this, vendors would
have to support it in order to sell Forths.
LSE has its strong points, but what would you fix in Forth94 to
provide most of the strengths of LSE?
Brad
John Doty
> On Jul 19, 1:39 pm, John Doty <j...@whispertel.LoseTheH.net> wrote:
>> Brad Eckert wrote:
>>> On Jul 18, 6:04 am, John Doty <j...@whispertel.LoseTheH.net> wrote:
>>>> Yet you didn't "get" STOIC.
>>> Is that kind of like not "getting" CATCH/THROW in the thread
>>> http://groups.google.com/group/comp.lang.forth/msg/b228b688cc3a3182?h...
>> I have no idea why that would make you think I don't "get" CATCH/THROW.
>>
>> Maybe an analogy would help: I own two chainsaws. I also own a bowsaw. I
>> tend to get the bowsaw out more than the chainsaws. Especially if I have
>> a helper without chainsaw experience.
>>
>
> Different application domains.
I think it's more differing perspectives.
> Elizabeth likes CATCH/THROW because she
> has some complex pieces of code that can experience I/O or other
> errors in multiple places.
She's a professional programmer, she likes to make that part easy. But
programming is already the easiest part of the software cycle from
project conception to decommissioning.
> But you have simpler error handling
> requirements so you use error flags.
Do I? Or do I generally choose simpler means because they cause less
trouble in the long run?
> Both you and Elizabeth understand
> that it's easy to abuse CATCH/THROW, I think.
Yeah, but what about that third party? You, know, the one who adds a new
feature without fully understanding the subtleties. Software has a life
after the programmer finishes.
> You're both right but
> you still bump heads. "Fascinating", as Spock would say.
>
> But back to the OT, building a new Forth, there is a question of which
> Forth to build. Yes, Forth usage is in decline. But it will never go
> away because it's too good at what it does.
Cue Al Kooper's "Going, Going, Gone".
Forth is too good at controlling astronomical instruments to go away. It
essentially has, anyway. Positive feedback: the less familiar it is, the
less it can be used, even by those who would like to use it.
Technologies don't die because they are unneeded. They die when they
lose "critical mass".
> Forth is words and stacks,
> as Chuck says. ANS Forth adds wordlists and restrictions to promote C-
> style portability.
It has failed to do so outside the protected environment of the
professional Forth shops.
>
> As I see it, the evolution of Forth isn't the problem. The evolution
> of reusable code (like libraries and applications) is the problem. A
> body of code with environmental dependencies will tend to encourage
> the Forth vendors to support those dependencies. For example, the FSL
> assumes a separate floating point stack so all the Forths support
> that. Much of my code deals with bytes and wouldn't work if chars
> weren't bytes, so I wantonly embrace that dependency and make sure all
> my targets support bytes. If a lot of code did this, vendors would
> have to support it in order to sell Forths.
Where do you think such code comes from? Most comes from users, not
programmers. Look at the explosive growth of the Python packages, and
look at who writes them. The benefits of consciously designing a
language for users who may not be CS experts.
>
> LSE has its strong points,
And I'll be the first to admit to weak points.
> but what would you fix in Forth94 to
> provide most of the strengths of LSE?
When Ritchie designed a systems programming language to replace PL/I, he
didn't start from PL/I, although the design of C was surely informed by
experience with Multics PL/I.
The very first decision I'd make for a new Forth design is alien to the
whole Forth94 approach: a Forth should either stick with the cell as a
universal data container (ColorForth, LSE) or have the machinery to
manage data types (StrongForth).
Michael
...
>interactivity and incremental compilation. For years I've used Pic assembly >language...
May be you want to take a look at amforth:
http://amforth.sourceforge.net/
Its kernel is written in assembler in a one-word-one-file concept from
the scratch. It was done by Matthias Trute for atmels ATmega familiy
and it is free. Mayby you can use it for pics too. Just redefine those
few forth code words of the kernel along with your i/O. As you say:
"Make it work, then make it fast..."
Viele Grüße, Michael
paultbar...@gmail.com
> Hi folks,
>
> I wanted to introduce myself and throw out some ideas in what I perceive
> to be as a gap in the Forth continuum. Any feedback would be
> appreciated. Sorry for the length but I've been doing a lot of thinking.
> Feel free to snip whatever you like. I have a lot of setup here, you can
> skip down about 150 lines if you are interested in only my questions.
>
> I'm an IT professor at Clayton State University in the Atlanta area. So
> my interest is primarily academic. I'm also a tool builder. The sense of
> understanding and control I get when working with my own tools is
> appealing. I'm also an Open Source and Linux guy. So I'll happily use
> others' tools so long as I have the ability to adapt them to my needs.
> Finally I'm a embedded systems guy that has been using Microchip Pics
> for development of small scale embedded systems projects for years. I'm
> well aware that the architecture is quirky and limited, but the
> confluence of price, documentation, and simplicity has made it a choice
> of mine for awhile. Plus Microchip offers free samples.
>
> So where does Forth fit into this? Truthfully I wish that I had come
> across it 15 or 20 years ago. The simplicity and elegance of Forth's
> language/system concepts (stack based, generating/incremental compilation,
> and interactivity) are intriguing. Forth came across my screen a couple
> of weeks ago while I was taking a look at the Parallax Propeller:
>
> http://www.parallax.com/propeller/index.asp
>
> A $12 8 CPU multiprocessor embedded system that has a collective 160
> MIPS of processing power and comes in a 40 pin DIP suitable for
> prototyping is a wonder to behold. But the culture around the part also
> illustrates why I like building my own tools. Parallax developed a high
> level object language called Spin around the part. They embedded an
> optimized bytecode interpreter for the language into the part. The
> disheartening thing is that they then build the compiler and IDE
> environment for Spin specifically for Windows. Worse is that the
> source and specifications for the environment are closed. In short to
> program the part in Spin you must use their tool on their platform.
>
> Sigh.
>
> Fortunately another developer, Cliff Biffle, who was in the same boat (a
> Mac OSX guy) solved the problem by hosting Forth on the chip.
> PropellerForth:
>
> http://www.cliff.biffle.org/software/propeller/forth/index.php
>
> requires only a terminal interface to get development done. In addition
> Cliff has added words to facilitate multiprocessor tasking. So it's
> possible to run multiple simulteaneous tasks on each of the 8 Propeller
> cogs (CPUs).
>
> Cliff's contribution got the ball rolling for me. Searching around let
> me pretty quickly to Starting Forth, Thinking Forth, and Steven Pelc's
> Programming Forth. I also installed gforth on my Linux box and started
> noodling.
>
> Fortunately for me stack based programming is old hat. Over the years
> I've built interpreted languages for processing web pages, teaching
> beginning programming, and programming PICs (combination of HLL compiler
> along with a bytecode interpreter). In each I utilized the concept of an
> execution data stack for processing expressions and subroutine calls and
> parameters (when I implemented subroutines). But each time I approached
> the high level language in the traditional compilation, parsing, code
> generation/interpretation view of most high level languages. Forth's
> view of the world instantly clicked with me.
>
> So the next question for me was "What about forth for PICs?". As a long
> time contributor to the Piclist mailing list, I can never remember seeing
> a Forth discussion. Google of course popped out a handful of what are
> the usual suspects: Picforth: http://www.rfc1149.net/devel/picforth and
> Mary: http://mary.pepix.net. I'm limiting my discussion to Forths for
> the 16F PIC family as that's my primary focus.
>
> I played with Picforth for a few and quickly realized that it was in
> fact a Forth compiler. I was disappointed because I feel that takes away
> one of Forth strengths: interactivity and incremental compilation. For
> years I've used Pic assembly and other languages such as Jal, and my own
> PIC HLL called NPCI for development. As compilers you fall into the
> traditional edit -> compile -> download -> test cycle. That's exactly
> the cycle I wish to get away from. The ideal Forth for embedded
> development would be interactive, with incremental development, and self
> hosted so that when the application is finished, it's all onboard the
> target. So the game's afoot!
>
> c.l.f is a rich resource of discussion on the subject. Searches for PIC
> Forth, minimal forth, target forth, and embedded forth came up with mega
> threads of discussions that I've been pouring over. A couple of
> observations:
>
> 1. A specification for hosting Forth with a minimal kernel would be
> instructive for tool developers because it would abstract what one needs
> to implement Forth in a particular environment. One of my maxims to my
> students is "Make it work. Then make it pretty." In Forth's case I'd
> change that to "Make it work. Then make it fast." The concept of minimal
> kernels always gets shot down because of the resulting speed of the
> result. However, if a developer can get Forth working on the target,
> then optimizing words to make it faster can be done incrementally as you
> move forward. One crack at this was done in this thread:
>
> http://tinyurl.com/22c7sn
>
> with the specification here:
>
> http://www.quirkle.com/misc/forth.htm
>
> While I think Jim has the wrong target audience, the general Forth
> enthusiast, the idea has merit if targeted towards tool builders.
>
> 2. Such a specification should be architecture neutral. Specs for
> eforth, hforth, and retroforth (which are often suggested when the
> question "How do I build my own Forth?" come up) that I've seen seem to
> be a bit too tightly coupled to the X86 platform. A forth specification
> that factors out the primitives would be helpful. A tool builder is
> probably going to target something that has not in fact been done yet.
>
> 3. For the purposes of embedded development, that an environment that
> appears as close to self hosting as possible would be desirable.
>
> Frank Sergeant idea is really close to being the jackpot. What could be
> a simpler kernel than 3 instruction Forth?
>
> http://pygmy.utoh.org/3ins4th.html
>
> It's almost like Frank was thinking about me coming along 16 years later
> when he wrote this. Elizabeth gives an explanation of the tethered Forth
> approach to embedded systems development here:
>
> http://tinyurl.com/yv6z99
>
> In evaluating Frank's discussion I perfectly agree with why the
> microkernel approach is the right one, put the simplest, easiest to test
> executive on the target will get you going the fastest. Couple that with
> a full fledged forth environment on the host, you can get rolling pretty
> quickly.
>
> However, I disagree with Frank's assertion that you actually have Forth
> on the target. There's a gap between Frank's bytecode to execute
> arbitrary code on the target and actually running Forth on the target.
>
> So at long last we get to the gap I referred to in my opening. Frank's 3
> instruction executive facilitates getting an embedded system connected
> to a Forth host so you can interactively develop forth on the host and
> exercise the target. But his XCALL instruction facilitates calling
> arbitrary code on the target, not Forth code.
>
> So the question is "Given an embedded micro, what is the minimum forth
> environment required to execute Forth words?". If we can answer this
> question then along with the memory read/write instructions it should be
> possible to migrate Forth words (both code and compiled) to the target
> and have them executed there. If you can incrementally migrate the
> application to the target while developing it, then when you finish you
> can simply disconnect the target from the host and run the entire
> application on the target. All the while during development, you have
> the full facilities of the hosted forth environment available for you to
> interactively develop the application.
>
> Brad Rodriguez's Moving Forth articles gives a pretty good idea of how
> to pull it off:
>
> http://www.zetetics.com/bj/papers/moving1.htm
>
> Brad is of course very interested in how to implement such a kernel in
> an optimized way. But again "make it work, then make it fast." comes to
> mind. In my estimation one needs two sets of items to execute forth words:
>
> 1. The forth virtual machine including the standard registers and
> stacks.
>
> 2. Three critical words: ENTER, EXIT, and NEXT.
>
> In short create enough Forth to execute the inner interpreter and you
> can get going.
>
> It seems to me that neither of those two items are so complicated that
> they could not be easily put together for a target. And Frank's 3 word
> forth implements a inner-inner interpreter that can be used to create
> the inner interpreter required to run forth words on the target.
>
> Only one piece of the puzzle is missing at this point. Elizabeth
> discusses in her post above that the XTL transfers the stack between the
> host and the target. In short it implements a form of distributed
> execution where you muster the stacks for RPC. In doing so one can run a
> application with a set of words distributed between the host and the
> target.
>
> So I envision an environment where Forth words are quick interactively
> developed on the host. Once satisfied they work, they are incrementally
> compiled on the host and transferred to the target. applications can be
> a set of shared words between the two until complete at which point all
> needed words are transferred to the target, completing the app.
>
> There are still details that need to be worked out such as how to
> differentiate between local words and remote words in both systems and
> how to facilitate transferring the stacks between the two. In both cases
> solutions should be geared towards simplifying the target.
>
> I do believe that the minimal kernel still needs to be specified. It'll
> give a list of words that needs to be implemented, tested, and
> incorporated in the initial kernel. I can help here because as I stated
> earlier my NPCI bytecode interpreter implements a stack based virtual machine
> and is written in pic assembly for the 16F family. It has debugged code
> for all 16 bit arithmetic and logical ops, number processing for stacks,
> and conditional/unconditional jumps. It already implements an IP for the
> bytecode.
>
> I look forward to hearing your comments on these thoughts.
>
> TTYL,
>
> BAJ
I have used Bryte-Forth for the Intel-8031 family, still have the manual and code.
Would love to see a FORTH for the PIC maybe the 18F family.
IdeZilla
Mikael Nordman
> I have used Bryte-Forth for the Intel-8031 family, still have the manual and code.
> Would love to see a FORTH for the PIC maybe the 18F family.
>
> IdeZilla