Looking for Forth with special capabilities.

[Cecil - k5nwa]

I want to work on a special educational project so I'm looking for a
Forth with special qualities. I will use this software to build other
Forth Systems with optimizing capabilities.

It needs to be able to generate a stand alone target program that has a
minimum code in it so it should not include the Forth System in it just
the desired code. So in essence it's a target compiler for the Pentium
Family of CPUs.

If that is not available an alternative is that it should be able to
generate a library file without Forth inside it just the application
code. Then I can use a C wrapper to execute the software. In some ways
being able to generate an application only library has a few benefits of
it's own.

The host system will be a Linux system running specifically Mint 19.3
x64 Linux. Currently I have been using C in Windows but I would like to
move away from C and Windows if possible, I'm getting to the point where
I need to create some complex libraries in C but I rather do it in Forth
if possible. Extra points will be awarded for a 64-bit solution as Linux
systems are moving away from 32-bits

I have SwiftForth, and VFX Pro but to use them I have to drag the whole
Forth development system along and I do not wish to do that if
avoidable. Both of these include undocumented target compilers so that
they are hard to work with if not impossible, and they are written so
basically the entire Kernel needs to be included in the application.

Any ideas on some Forth system that might be usable for this special
purpose? Otherwise I will have to continue using C or switch to
something like Lazarus which is more user friendly than C due to built
in target Graphical IDE components and error checking.

Another remote possible system is Timbre which could be used to generate
an assembler file that is stand alone but it's an older Windows only
Forth with little documentation so there will be some difficulties but
it also has some nice advantages such as it's superb text manipulation
capability..

Thanks for your suggestions.
--
Cecil - k5nwa

[none albert]

In article <r6ng17$841$1...@gioia.aioe.org>,

Cecil - k5nwa <cba...@cbayona.com> wrote:
>I want to work on a special educational project so I'm looking for a
>Forth with special qualities. I will use this software to build other
>Forth Systems with optimizing capabilities.
>
>It needs to be able to generate a stand alone target program that has a
>minimum code in it so it should not include the Forth System in it just
>the desired code. So in essence it's a target compiler for the Pentium
>Family of CPUs.
>
>If that is not available an alternative is that it should be able to
>generate a library file without Forth inside it just the application
>code. Then I can use a C wrapper to execute the software. In some ways
>being able to generate an application only library has a few benefits of
>it's own.
>
>The host system will be a Linux system running specifically Mint 19.3
>x64 Linux. Currently I have been using C in Windows but I would like to
>move away from C and Windows if possible, I'm getting to the point where
>I need to create some complex libraries in C but I rather do it in Forth
>if possible. Extra points will be awarded for a 64-bit solution as Linux
>systems are moving away from 32-bits
>
>I have SwiftForth, and VFX Pro but to use them I have to drag the whole
>Forth development system along and I do not wish to do that if
>avoidable. Both of these include undocumented target compilers so that
>they are hard to work with if not impossible, and they are written so
>basically the entire Kernel needs to be included in the application.

There is ciforth and "dragging the whole Forth" along is a mere 50K
even for 64 bit. "Target compilation" is an adjustment of the size
of the Forth and even if it could be better documented, there is
just one screen to decipher.
Of course the whole Forth consist for 80% of indispensable code like
@ ! and OPEN-FILE.

You can couple to ciforth using SYSTEM and channels (READ-FILE
WRITE-FILE) instead of going through dynamic libraries and complicated
object formats.
It can be that the main system is ciforth too. Then you would
just use the multi-tasking that sports actual separate processes.

>
>Any ideas on some Forth system that might be usable for this special
>purpose? Otherwise I will have to continue using C or switch to
>something like Lazarus which is more user friendly than C due to built
>in target Graphical IDE components and error checking.
>
>Another remote possible system is Timbre which could be used to generate
>an assembler file that is stand alone but it's an older Windows only
>Forth with little documentation so there will be some difficulties but
>it also has some nice advantages such as it's superb text manipulation
>capability..
>
>Thanks for your suggestions.
>--
>Cecil - k5nwa

Groetjes Albert
--
This is the first day of the end of your life.
It may not kill you, but it does make your weaker.
If you can't beat them, too bad.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Pablo Hugo Reda]

Try :r4 for 32 bit
can generate standalone without forth interpreter

https://github.com/phreda4/reda4

[Cecil - k5nwa]

50KB is not bad that is a small Kernel, my current one is 16KB with
close to 200 words it small but at the cost of speed, I understand a lot
of basic words are need to be carried along to execute the application
is just that with the commercial offering that includes a lot of
unneeded baggage. I have been curious about ciforth. At this point my
ideal system would generate machine code or better yet assembler code in
ASCII so I can look at it and see what is happening, an ASCII assembler
file is my preference so I can use code to manipulate the text and
optimize it. Even though Timbre is poorly documented it has some
terrific text manipulating capabilities that are rather nice.

In the past I've written several optimizing Forth target-compilers but
I've done it in unconventional ways by optimizing the generated ASCII
file, why? Because I understand the process well and even though it not
the traditional way it works for me.

One advantage with something like VFX is that it has an awesome compiler
so it generates excellent code totally the opposite of what I have
currently done which is a token threaded Forth system in C but I'm ready
to dump it and try something new.

>
> You can couple to ciforth using SYSTEM and channels (READ-FILE
> WRITE-FILE) instead of going through dynamic libraries and complicated
> object formats.
> It can be that the main system is ciforth too. Then you would
> just use the multi-tasking that sports actual separate processes.
>

I have no objection to system libraries, there are some advantages to
generating some.

>>
>> Any ideas on some Forth system that might be usable for this special
>> purpose? Otherwise I will have to continue using C or switch to
>> something like Lazarus which is more user friendly than C due to built
>> in target Graphical IDE components and error checking.
>>
>> Another remote possible system is Timbre which could be used to generate
>> an assembler file that is stand alone but it's an older Windows only
>> Forth with little documentation so there will be some difficulties but
>> it also has some nice advantages such as it's superb text manipulation
>> capability..
>>
>> Thanks for your suggestions.
>> --
>> Cecil - k5nwa
>
> Groetjes Albert
>


--

Cecil - k5nwa

[Cecil - k5nwa]

Any plans for Linux? Like I said, I'm moving away from Windows and any
new projects need to use Linux unless it's impossible.

Being based on ColorForth does it use the funky Dvorak keyboard that
many versions use?

I keep finding and keep being disappointed on how complex some of the
modern Forth are, to me it looses some of the virtues of Forth being
simple to use and understand but ColorForth has it's issues in funky
keyboard, tiny display, etc, I think Charles Moore made it too personal
and forgot about the rest of us. For a while I tried finding more
information on Machine Forth but there is little out there and there are
lack of implementations. My favorite of his is cmForth some nice
improvements and simplifications, for years I used Pygmy which had a
built-in target compiler and was based on cmForth.

I'll look at :r4 in the next week or so and see if it will be useful for
my project.


--
Cecil - k5nwa

[Pablo Hugo Reda]

Through WINE it works correctly in linux.

The current development version called r3 is 64 bits, using the SDL library so it could be compiled in linux.

It is based on the working ideas of ColorForth, not IDE, so it uses the standard keyboard and the file system.

Good luck

[Cecil - k5nwa]

On 4/9/20 11:28 AM, albert wrote:

I just downloaded ciforth from Github along with Lina and Wina.

Just a cursory reading, ciforth is a Forth system generator? You tell it
what you want and it generates a forth system, bit-size, OS, OS
interface type, it sounds like it will be an interesting piece of
software. I will be reading and printing the documentation today and
experimenting with it tomorrow. First build might be something a system
for Linux, 64-bit, using system calls.

I see some m4 files in there is that M4 the macro processor, I'm not
familiar with it but I've used powerful macro-processors before to
create an optimizing target Forth compiler for several targets back in
the 70s' and it worked well it's not traditional at all, the important
thing was that it generated correct efficient code, one thing I like was
that it was easy to incrementally improve and migrate the code
generating code.

I could be off since I haven't gone into it much but that approach is
why I like Timbre, it has an extremely powerful easy to use macro system
built in, unfortunately the documentation is lacking for it.

--
Cecil - k5nwa

[none albert]

In article <r6qbhv$t25$1...@gioia.aioe.org>,

Right! ciforth is is a Forth system generator.
You don't want to use that, unless you're into a new processor like
RISC-V or want a booting 64 bit system directly from uefi.
It is complicated because it generates something that is
as simple as possible.
See also:
https://github.com/albertvanderhorst/ciforth/wiki/Philosophy-behind-ciforth

Just download the lina64###.tgz It is 3 files:
executable, library, documentation.
(Then there is an asm/s source file if you want to.
Then there is a one line command to make the executable from
the source if you want to.)

>
>I see some m4 files in there is that M4 the macro processor, I'm not
>familiar with it but I've used powerful macro-processors before to
>create an optimizing target Forth compiler for several targets back in
>the 70s' and it worked well it's not traditional at all, the important
>thing was that it generated correct efficient code, one thing I like was
>that it was easy to incrementally improve and migrate the code
>generating code.
>
>I could be off since I haven't gone into it much but that approach is
>why I like Timbre, it has an extremely powerful easy to use macro system
>built in, unfortunately the documentation is lacking for it.

I hope it is clear that for your purpose you don't need to dive
into the m4 stuff. Although m4 is not an easy tool at least it
is completely documented.

>
>--
>Cecil - k5nwa

Groetjes Albert

[Cecil - k5nwa]

Thanks.

I tried to find information on Machine Forth and some on Color Forth but
there seems to be a limited amount of information.Al least Machine Forth
seems to be very small if I found the full source, it's only a couple of
pages of code.

--
Cecil - k5nwa

[Cecil - k5nwa]

Thanks, Ive downloaded ciforth, Lina and Wina. It's that kind of system
I want to play with and experiment with.

I'm not all familiar with M4, mostly I've used Stage2 a DOS base
macro-processor, it is deceiving, it looks primitive but it can do just
about anything, I implemented a multi-CPU optimizing target compiler
with it alone back in the 70s'.

I also experimented with Gema, and with Timbre, Gema is more modern it's
a Windows Console Application, but Timbre is not only an extremely
capable macro-processor but it has full access to Forth so one can do
just about anything one wants, it's a Windows Console application that
runs fine in Wine, Gema works fine in Wine too, and does Stage2.

To me all this macro stuff is very fascinating and very easy to
understand and that is basically what I want to play with.

I have the source to Timbre, Gema, and Stage2 so porting to Linux is a
possibility, Stage2 is highly portable, the other two I'm not so sure.
In any case since they run in Wine and their output are text files,
there is little need to port anything.

--
Cecil - k5nwa

[Cecil - k5nwa]

Looks like there are several possibilities available for me to try;

Albert's ciforth
Pablo's :r4
Andreas MinForth

Those three aught to keep me out of trouble during these days of
voluntary home confinement.

Thanks fellows for the ideas, I will be checking them out as my health
allows. Having headaches and dizziness 24/7 makes for slow progress.


--
Cecil - k5nwa

[A. K.]

Thanks and get well soon!
Andreas

[Stephen Pelc]

On Thu, 9 Apr 2020 10:46:48 -0500, Cecil - k5nwa <cba...@cbayona.com>
wrote:

>I want to work on a special educational project so I'm looking for a
>Forth with special qualities. I will use this software to build other
>Forth Systems with optimizing capabilities.
>
>It needs to be able to generate a stand alone target program that has a
>minimum code in it so it should not include the Forth System in it just
>the desired code. So in essence it's a target compiler for the Pentium
>Family of CPUs.

Many, many years ago, 1991/2, MPE shipped a 16 bit DTC Forth called
PC PowerForth Plus. It came with a package called a "code stripper".
When you told it which word was run at cold start, it produced a
minimum size executable. For a simple hello world, the DOS executable
was 452 bytes long.

The code was written by Sergei Baranov.

Porting this to 32 and 64 bit Forths would be a project, but perhaps
of value to more than one person.

If you want to take this further, perhaps we can sort out the IPR
issues with Sergei offline.

Stephen

--
Stephen Pelc, ste...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612
web: http://www.mpeforth.com - free VFX Forth downloads

[none albert]

In article <5e9599c2...@news.eternal-september.org>,

Stephen Pelc <ste...@mpeforth.com> wrote:
>On Thu, 9 Apr 2020 10:46:48 -0500, Cecil - k5nwa <cba...@cbayona.com>
>wrote:
>
>>I want to work on a special educational project so I'm looking for a
>>Forth with special qualities. I will use this software to build other
>>Forth Systems with optimizing capabilities.
>>
>>It needs to be able to generate a stand alone target program that has a
>>minimum code in it so it should not include the Forth System in it just
>>the desired code. So in essence it's a target compiler for the Pentium
>>Family of CPUs.
>
>Many, many years ago, 1991/2, MPE shipped a 16 bit DTC Forth called
>PC PowerForth Plus. It came with a package called a "code stripper".
>When you told it which word was run at cold start, it produced a
>minimum size executable. For a simple hello world, the DOS executable
>was 452 bytes long.
>
>The code was written by Sergei Baranov.
>
>Porting this to 32 and 64 bit Forths would be a project, but perhaps
>of value to more than one person.
>
>If you want to take this further, perhaps we can sort out the IPR
>issues with Sergei offline.

This would be a nice little exercise on ciforth. The practical value
of shrinking a 50 kbyte executable to 5 kbyte is dubious on a
64 bit system with a 3 Tbyte hard disk, though.

Design :
1. recursively mark all definitions that are used
2. keep a source and destination pointer
initialise both at the lowest definition
prepare a table of pairs
3.
from lowest to highest
if the definition is not in use
increment source pointer
else
add the pair (source, destination) to a table
for a high level definition patch all addresses
according to table
copy the definition from source to destination
update both source and destination pointers
4. set destination pointer to DP (HERE) and the
start address in the ABORT vector and
write the image to file.

Most practical code can be borrowed from the decompiler and the
turnkey screens.

Interestingly, it is probably nicer to just inline everything recursively
until code definitions are reached. Then the monolithic code can be
copied over the old Forth. This may expand a 50K program to 200K
but it will have a speed advantage. Repetitive code is offset
by the lack of headers.
I do something like that in the optimiser I'm working on.
LEAVE and EXIT are tricky and must be replaced by branches
before expansion, but that is the only pitfall.
Keeping track of where the branches go after expansion is
the tedious part.

>
>Stephen

[dxforth]

> …

Much of the bulk of a Forth system is the compiler & tools. Organize
it such that the latter can be easily jettisoned and the run-time drops
to a few KB. If C apps manage without lugging a compiler around with
them, why not Forth :)

[Alex McDonald]

<discuss>

Because you've just thrown away the whole of Forth, all for a few bytes.
Forth's run time is source.

</discuss>

--
Alex

[Stephen Pelc]

On 14 Apr 2020 12:17:39 GMT, albert@cherry.(none) (albert) wrote:

>This would be a nice little exercise on ciforth. The practical value
>of shrinking a 50 kbyte executable to 5 kbyte is dubious on a
>64 bit system with a 3 Tbyte hard disk, though.

It also applies to RAM. If a minimally equipped PC has 1 Gb of RAM,
then there's very little point in worrying about a Forth executable of
30 Mb, which corresponds to 3% of RAM. For a PC with 4 Gb of RAM, this
becomes less than 1% of RAM.

The only area in which size can impact is in the cost of software
downloads and updates.

[Cecil - k5nwa]

On 4/14/20 6:28 AM, Stephen Pelc wrote:
> On Thu, 9 Apr 2020 10:46:48 -0500, Cecil - k5nwa <cba...@cbayona.com>
> wrote:
>
>> I want to work on a special educational project so I'm looking for a
>> Forth with special qualities. I will use this software to build other
>> Forth Systems with optimizing capabilities.
>>
>> It needs to be able to generate a stand alone target program that has a
>> minimum code in it so it should not include the Forth System in it just
>> the desired code. So in essence it's a target compiler for the Pentium
>> Family of CPUs.
>
> Many, many years ago, 1991/2, MPE shipped a 16 bit DTC Forth called
> PC PowerForth Plus. It came with a package called a "code stripper".
> When you told it which word was run at cold start, it produced a
> minimum size executable. For a simple hello world, the DOS executable
> was 452 bytes long.
>
> The code was written by Sergei Baranov.
>
> Porting this to 32 and 64 bit Forths would be a project, but perhaps
> of value to more than one person.
>
> If you want to take this further, perhaps we can sort out the IPR
> issues with Sergei offline.
>
> Stephen
>

That would be great, since VFX generates very good code with something
like that one could generate very small stand alone applications which
is one of the things I'm interested in. Why small
? So it can fit in the cache of the CPU and be lightning fast among
other things, I'm experimenting with Forth compilers and that would make
sure that I'm just generating the application code and nothing else.

Given a preference I would prefer a Linux version first.

Thanks


--
Cecil - k5nwa

[The Beez]

On Thursday, April 9, 2020 at 5:46:55 PM UTC+2, Cecil - k5nwa wrote:
> If that is not available an alternative is that it should be able to
> generate a library file without Forth inside it just the application
> code. Then I can use a C wrapper to execute the software. In some ways
> being able to generate an application only library has a few benefits of
> it's own.
>
> The host system will be a Linux system running specifically Mint 19.3
> x64 Linux. Currently I have been using C in Windows but I would like to
> move away from C and Windows if possible, I'm getting to the point where
> I need to create some complex libraries in C but I rather do it in Forth
> if possible. Extra points will be awarded for a 64-bit solution as Linux
> systems are moving away from 32-bits

What you're describing is 4tH (google: 4th compiler). It creates a bytecode image which you can run from the compiler *OR* you use a Makefile and it will create an executable for you. What you link in is just the runtime. Compiler, Decompiler, Preprocessor are separate - and you don't need that one for execution.

It does 32bit, 64bit, Linux, Windows, OS/X, DOS, Coherent, BSD and probably a lot other *nixes.

Hans Bezemer

[Cecil - k5nwa]

I'll look at it, I have a lot of time on my hands and even though I have
serious health issues I can read and do small experiments to try to work
things out.

My preference at the moment is for the software to generate assembler
code or C as output, I'll take either one so I can look at it, make
changes if needed and end with a small footprint in code size. A Forth
code stripper like Steven Pelc mentioned is also a highly workable
solution and has other uses in creating applications.

--
Cecil - k5nwa

[Cecil - k5nwa]

On 4/14/20 6:28 AM, Stephen Pelc wrote:

> On Thu, 9 Apr 2020 10:46:48 -0500, Cecil - k5nwa <cba...@cbayona.com>
> wrote:
>
>> I want to work on a special educational project so I'm looking for a
>> Forth with special qualities. I will use this software to build other
>> Forth Systems with optimizing capabilities.
>>
>> It needs to be able to generate a stand alone target program that has a
>> minimum code in it so it should not include the Forth System in it just
>> the desired code. So in essence it's a target compiler for the Pentium
>> Family of CPUs.
>
> Many, many years ago, 1991/2, MPE shipped a 16 bit DTC Forth called
> PC PowerForth Plus. It came with a package called a "code stripper".
> When you told it which word was run at cold start, it produced a
> minimum size executable. For a simple hello world, the DOS executable
> was 452 bytes long.
>
> The code was written by Sergei Baranov.
>
> Porting this to 32 and 64 bit Forths would be a project, but perhaps
> of value to more than one person.
>
> If you want to take this further, perhaps we can sort out the IPR
> issues with Sergei offline.
>
> Stephen
>

Thinking about it a little bit, your company having the Vfx Target
compiler to generate stand alone applications is a great start to
something like what I'm looking at, with code stripping one could end up
with very small code.

Actually using your Target compiler would be a very workable situation
if it were documented, and it might be but I have not found the
description on how to create an application that it can compile and
generate the code without the Forth Compiler inside it.

I looked at it some and since Vfx is pretty complicate I was not able to
sort it out and figure it out but it's another thing to research.

--
Cecil - k5nwa

[m...@iae.nl]

Many, and I mean many many, moons ago I was also fascinated by
generating minimum size and superfast executables out of Forth
source. I started with TCOM, and ended up with my own metacompilers
for Z80, 68K, and the TMS320C40 dsp. At that time I also
owned FysForth on a PC, and that Forth also was big on
having machine code libraries.

However, at some point I noticed that I was actually wasting
my time with endless recompiles of the whole system, every
time that I noticed a small bug or imperfection in the modules
or binaries. Also, I constantly went back to the original
source code to eyeball it for the bugs or imperfections
that I couldn't find anymore by interactively debugging
and inspecting the targets.

I drew the conclusion that by turnkeying (but not by
making it superfast :-) I was actually throwing away
the characteristics that made Forth such a nice language
and a joy to use. This is all very much IHMO, because
some co-developers of iForth have a completely different
opinion. At least, for now :-)

-marcel

[A. K.]

My similar experience: with over-optimization comes complexity;
complexity is an unwanted intellectual burden and makes everybody
make mistakes; mistakes are expensive and make everybody unhappy.

So we developed the key questions:
- is that thing maintainable by non-specialists?
- if specialists' support is required, can we do it quickly and remote?
- does it require complex tools?

In other words: optimize for simplicity and fast work results,
never for execution speed or code size (unless there are stringent
technical reasons).

[Peter Forth]

Of course , this is absolutly correct !

IMO
A Forth without an interpreter is like asking a vanilla Ice without the
vanilla...

I would recommend : programm on anything else, but not forth ! there
is no advantage to start with forth if the only point is chasing for
speed. In that case use the forth-Macro-asm to write code.

We exchanged in the past about this, commenting about Tcom
and NCC of LMI (the native code compiler). Wasn´t you Cecil
telling me that Pigmyforth had also a minimal binary generation option ?

Even NCC needed the interpreter when generating the turnkey/standalone.
It was able to produce very small binaries, at aprox. 40K.
(There was an option of 20k headless code kernel).

For today standarts with the smallest pendrives working on the Gigabyte
range 20K 40K 50K is meaningless. Even expecting a forth program
to fit in a cpu cache, at some moment accessing external data or a driver,
implies to jump out to external memory.It will end at the same speed
as any other modern stc forth , but probably much more difficult
to use.


Tom Almy has also this one
The ForthCMP Forth Compiler
ForthCMP is the Forth language compiler for computers running MS-DOS. It has the following major features:

Compiles Forth into machine code -- not interpreted
Written in Forth so that Forth code can be executed during compilation
Very fast compilation -- compiles into an executable file in a single pass
Very compact -- over 110 primitive words compiled in-line. Many optimizations
Built-in assembler
Libraries in source format
Memory models include COM, EXE, SYS, and ROM-able
Many demonstration programs provided
You simply won't believe the small size of the compiler or the programs that it generates. With today's systems compilation is "instantaneous." This compiler represents mature, stable technology. The first version was produced in 1982 and it has been enhanced over the years for better operation.

See a demonstration of "Hello World" here.

ForthCMP comes in two versions, one for the ANS Forth Standard, and the other supporting the older 1983 Forth Standard. This shareware program is fully functional, only the documentation is withheld. However there are instructions for compiling the demonstration programs. Wondering about ANS Forth Standard compliance and optional extension word sets? Check here.


Download the ANS Forth version 2.3s (111k) REVISED 01/1999
Download the 1983 Standard Forth version 2.21o (98k) REVISED 12/1998

The distribution method for ForthCMP has changed. The $50 registration fee now licenses both versions. When you register, you the documentation for both versions and the ANS Forth draft standard as PDF files via EMAIL if you order vial Paypal or on CD if you order via mail (same price but takes longer!).

You can register via mail using the instructions in the READ.ME file in the downloaded archive, or you can pay for your registration using PayPal for faster delivery. If you use Paypal, you can charge the purchase to your credit card. Click on the icon below to order:

PayPal - The safer, easier way to pay online!
This game, Sokoban (14k), is written in ForthCMP. The source code is provided in the ANS Forth version, above. You don't need the compiler to run the program, but I'd suggest you download the compiler as well to see how it works.
Everything you might want to know about Forth can be found at this Forth Information Page. It includes ANS Forth documentation, several Forth primers, and other Forth implementations.

Y2K Compliance
Both versions of of ForthCMP available here, as well as all earlier versions, are Y2K compliant. The ANS Forth version implements the only function that utilizes dates, TIME&DATE, which returns a four digit date. As this is based on an operating system call, if the operating system is Y2K compliant, then ForthCMP will display the correct date.

Tom Almy
webma...@almy.us

[dxforth]

While forthers philosophize, those outside continue to write applications.

[none albert]

In article <r6ng17$841$1...@gioia.aioe.org>,

Cecil - k5nwa <cba...@cbayona.com> wrote:

I though I would follow up the original question with a description how
the experimental optimiser in ciforth currently works.

Optimising the ciforth way goes as follows.

This is the code we want to optimise:
(The infamous prime sieve, the unadulterated byte version)
-----------------------------------------
( DO-PRIME-ISO GILBREATH's_benchmark_MMI_mar17__VERSIE_#2)
\ Note by AH:
\ There is no specification, which is a defect in itself.
\ Approximately: it counts the primes up til 2^14
\ SIZE[N] contains a flag for 3+2N
8190 CONSTANT SIZE
CREATE FLAGS SIZE ALLOT

: SIEVE
FLAGS SIZE 1 FILL
0 ( 0 COUNT ) SIZE 0
DO FLAGS I + C@
IF I DUP + 3 + ( DUP . )
DUP I +
BEGIN DUP SIZE <
WHILE 0 OVER FLAGS + C! OVER + REPEAT
DROP DROP 1+
THEN
LOOP ;

--------------------------------------
There is an analyser that has extended headers for
Forth words which includes stack effect: lina-ana64
which is a saved system. The assembly code snippets
within ciforth are disassembled using code from ciasdis,
stack effects are analysed and exceptions added.
Then there is an optimiser.frt that loads on top of
lina-ana64.
In lina-ana64 you type
"
INCLUDE sieve.frt

'SIEVE OPTIMISED
'SIEVE code$ 2DUP "sieve64.bin" PUT-FILE
OVER + ASSEMBLER DISASSEMBLE-RANGE
"
You now have the binary code for the sieve in sieve64.bin
which you could use like
"sieve64.bin" GET-FILE CODE sieve ,, END-CODE
if it were not for the buffer addresses that change.

You can catch the output of the disassembly.
lina-ana64 | tee sieve.asm
add some CODE-HEADER stuff to sieve.asm
and just do
"
WANT ASSEMBLERi86 ALIAS
ASSEMBLER
'NEXT, ALIAS NEXT
: !TALLY ;
INCLUDED sieve.asm
: test version 10000 0 DO sieve DROP LOOP ;
"

lina-ana64 is an enhanced forth and retains the ability
to make turnkeys using the -c option:

-------------- citesta.frt 8<--------------------
WANT ASSEMBLERi86 ALIAS
ASSEMBLER
'NEXT, ALIAS NEXT
: !TALLY ;
"sieve.asm" INCLUDED
: test version 10000 0 DO sieve DROP LOOP ;
--------------------------8<--------------------

lina-ana64 -c citesta.frt
generates a turnkey citesta which is 300K because it contains
an analyser and an optimiser.
That is the point I'm currently at, and citesta is a speedup
of a factor 5.

In the future
lina64 -c citesta.frt
should work, but for now the i86 assembler in blocks
misses some 64 bit features.
The assembler can be loaded high. This means that after
assembling it can be trimmed. The resulting citesta would
be only slighly larger than lina64 itself, say 50 K.

>
>Thanks for your suggestions.
>--
>Cecil - k5nwa

[The Beez]

On Tuesday, April 14, 2020 at 7:51:17 PM UTC+2, Cecil - k5nwa wrote:
> My preference at the moment is for the software to generate assembler
> code or C as output, I'll take either one so I can look at it, make
> changes if needed and end with a small footprint in code size. A Forth
> code stripper like Steven Pelc mentioned is also a highly workable
> solution and has other uses in creating applications.

Well, it punches out C code - and it is able to pass information to and from C. You can call a Forth word (given you provide the proper parameters and retrieve the single return value) as a C function. It is entirely sandboxed and can't be crashed unless you make your hands dirty.You can even make it a kind of generator function or coroutine by exiting with PAUSE.

I've written a simple cooperative multitasker in C, where parameters are passed between C and 4tH using a shared stack. So 4tH puts its parameter on this stack, goes to sleep (effectively passing control back to C), C retrieves these parameters, does its thing, puts its parameters on the stack and passes the control back to 4tH. The whole multitasker program is less than 500 lines of C.

https://sourceforge.net/p/forth-4th/code/HEAD/tree/trunk/4tsh.c

Hans Bezemer

[a...@littlepinkcloud.invalid]

There have always been two distinct ways of thinking about Forth, and
writing Forth programs. One school tends to compile the Forth program,
then run it, without any runtime use of the dictionary and the
interpreter. (I've sometimes mischeivously referred to this as
"Reverse-Polish C.") The other recognizes that Forth is at heart a
dynamic programming language capable of runtime code generation,
interpretation, and adaptation, and uses all these tools in a sort-of
LISPy way. The first, of course, is the only recourse available to
anyone generating entirely-statically-compiled binaries, especially
for embedded systems. It's very cool that Forth is flexible enough to
accomodate both of these usages; I don't know of any other language
that can so comfortably do that.

Andrew.

[Cecil - k5nwa]

On 4/14/20 7:44 PM, Peter Forth wrote:
> On Tuesday, April 14, 2020 at 7:14:36 PM UTC-3, m...@iae.nl wrote:

SNIP

In their day Pygmy and ForthCMP were very useful tools capable of
creating standalone programs, and I like Pygmy a lot and used it for
years but and it's a huge but, they are DOS programs and generate code
for DOS 16-bit code at that, a little too old for what I want but what
they did I'm interested in.

As far as resources are concerned my typical "older small" PCs are are
quite capable, I've set aside one such PC to play with this project, it
has Linux Mint 19.3 16GB of RAM, SSD Hard Disk, and a Quad Core Q9650
3GHz CPU so any modern Forth would not remotely tax the system, that is
not the point. That is a small system, my larger PCs and I have 6 of
them are far more capable with i7 CPUs, more RAM, and way faster CPUs.

Mind you this is a personal project for educational purposes rather that
being a practical in today's systems and what I want in the long run is
to generate x64 code for Linux which is a long long way from a DOS
application. So while in principle they have some of the features I
want, they don't have x64 or even 32 bit code generation which takes
them out of the running altogether. I've considered migrating Pygmy to
be and generate x64 code but it's a huge task and even the author did
not attempt it and started from scratch to make a modern version based
on Python.

I've already gotten several suggestions that could be useful so after I
get my taxes done which I forgot about I will look at some of these
suggestions, there are several options that seem useful and workable
from tiny systems like :r4 based on the minimal ColorForth to ciforth a
compiler generating system and others in-between.

Practicality is not the aim of this project, but I have a specific goal
for it to use it as a learning vehicle.

--
Cecil - k5nwa

[Paul Rubin]

a...@littlepinkcloud.invalid writes:
> It's very cool that Forth is flexible enough to accomodate both of
> these usages; I don't know of any other language that can so
> comfortably do that.

Lisp?

[dxforth]

"Reverse-Polish C" is an accurate description of ANS - the inevitable
consequence of standardization, portability and locals. It would
prompt Jeff Fox to say:

"I see ANSI Forth as [...] one more failed attempt to find the ideal
portable scripting language."

[a...@littlepinkcloud.invalid]

Really? I wasn't aware of that. Can't you always do eval and whatnot?
Or is there a fully-compiled subset with no dictionary in there?

Andrew.

[a...@littlepinkcloud.invalid]

dxforth <dxf...@gmail.com> wrote:

> "Reverse-Polish C" is an accurate description of ANS - the inevitable
> consequence of standardization, portability and locals.

Not really, no: the strengths and weaknesses of the languages are very
different. It's hard for me to find many similarities beyond the
inevitable ones that come from being standardized languages that try
to provide low-level access.

> It would prompt Jeff Fox to say:
>
> "I see ANSI Forth as [...] one more failed attempt to find the ideal
> portable scripting language."

Jeff Fox said many things, some of them unwise.

Andrew.

[Anton Ertl]

Cecil - k5nwa <cba...@cbayona.com> writes:
>Why small
>? So it can fit in the cache of the CPU and be lightning fast among
>other things

That's a frequent, but wrong meme about the effect of caches.
Programs that are larger than the cache do not necessarily suffer much
from cache misses. In particular, code that is not executed does not
cause cache misses; more precisely, code that is not executed can be
loaded into the cache because it is in the same cache line as code
that is executed. But that only makes a significant difference if the
code that is not executed is interleaved with the code that is
executed; this is not the case for the Forth text interpreter and
probably also a minor effect for the "libary" use of words from the
Forth system.

The other issue is that there are other influences on speed than just
the cache miss rate. I have seen a number of cases where Forth
implementors gave preference to other aspects over actual, measured
speed. However, instead of finger-pointing I will show an example
using Gforth, comparing two ways of calling gforth-fast:

"gforth-fast --ss-states=1 --no-dynamic" is direct-threaded code.

"gforth-fast --ss-states=1 --no-super" is the same, but with code
replication: For every occurence of a primitive in Gforth's threaded
code, it produces a copy of the native code of the primitive, and
let's the threaded-code pointer point to it. This naturally has a
larger code size, but the benefit is that it increases indirect branch
prediction accuracy. The actual instructions executed are the same
(except that they reside in different locations).

Here are the run-times for the small Gforth benchmarks on a Celeron
J3455 (Goldmont):

sieve bubble matrix fib fft
0.356 0.480 0.664 0.616 0.484 gforth-fast --ss-states=1 --no-dynamic
0.272 0.292 0.208 0.424 0.192 gforth-fast --ss-states=1 --no-super

The dynamically generated code (the copies of the primitives) consumes
the following memory (produced with --print-metrics):

code size = 0 gforth-fast --ss-states=1 --no-dynamic
code size = 728037 gforth-fast --ss-states=1 --no-super

The static code size (text size) is 129586 bytes for this gforth-fast.

As you can see, the overall code size grows a lot with --no-super. So
why is --no-super faster, and how much effect does the code size have
on the I-cache misses? perf stat gives (on the Celeron J3455):

--no-dynamic --no-super
6,008,064,938 3,201,082,805 cycles
5,134,925,257 5,139,793,724 instructions
197,629,656 25,528,348 branch-misses
406,889 614,996 L1-icache-load-misses

In the "instructions" line you see that both versions execute almost
the same number of instructions; --no-super executes 5M more: these
instructions are executed when copying the code.

In the "L1-icache-load-misses" you see that the number of instruction
cache misses increases from --no-super. However, these 208k I-cache
misses cost maybe 20 cycles or so each, for a total of about 4M
cycles. Note also that the I-cache misses do not grow linearly with
the code size; here the effect is small, but you may also experience
big effects. In general, cache effects are hard to predict; but also,
in general, caches work well.

By contrast, --no-super reduces the "branch-misses" (branch
mispredictions) by a 172M. If we assume that each misprediction costs
16 cycles (which is realistic), this explains the performance
difference of 2.8G cycles.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2019: http://www.euroforth.org/ef19/papers/

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>There have always been two distinct ways of thinking about Forth, and
>writing Forth programs. One school tends to compile the Forth program,
>then run it, without any runtime use of the dictionary and the
>interpreter. (I've sometimes mischeivously referred to this as
>"Reverse-Polish C.") The other recognizes that Forth is at heart a
>dynamic programming language capable of runtime code generation,
>interpretation, and adaptation, and uses all these tools in a sort-of
>LISPy way. The first, of course, is the only recourse available to
>anyone generating entirely-statically-compiled binaries, especially
>for embedded systems. It's very cool that Forth is flexible enough to
>accomodate both of these usages; I don't know of any other language
>that can so comfortably do that.

Not sure about "so comfortably", but getting rid of the text
interpreter/REPL/etc. seems to be very common for languages that have
one (at least for older ones from a time when memory was still
scarce). Various BASIC compilers (without REPL) and Borland Prolog
(which was also a big deviation from mainstream Prolog in other
aspects) come to my mind.

[m...@iae.nl]

On Wednesday, April 15, 2020 at 5:40:01 PM UTC+2, Anton Ertl wrote:
> Cecil - k5nwa <cba...@cbayona.com> writes:
> >Why small
> >? So it can fit in the cache of the CPU and be lightning fast among
> >other things

[..]

> By contrast, --no-super reduces the "branch-misses" (branch
> mispredictions) by a 172M. If we assume that each misprediction costs
> 16 cycles (which is realistic), this explains the performance
> difference of 2.8G cycles.

That is interesting. Is there some special instruction or
secret sequence that we can use to advance-warn the CPU that
we are going to branch or call somewhere soon?

I suppose that if the hardware knew that we are
going to jump to $xxxx it could already prefetch
the code there. A pattern like "pop ebx, jmp ebx,"
or "ret," etc. is common enough that the hardware
could speculate with some confidence that the value
at the top of the return stack is a good candidate
for an imminent IP change.

If such a mechanism were in place, it would mean
that we should avoid using the return stack for data
because this data effectively hides where we are going to
jump/return: "pop ebx, pop ecx, ret" would be a bad idea,
"pop ebx pop ecx ...<useful code>... ret" is then much
better. I guess that RISC V exploits the fact that the
BL instruction exposes the return address in the link
register.

I would appreciate your insights.

-marcel

[Peter Forth]

I know this is your personal project as hobby, in that aspect is cool
you are enjoing the road and investigating diff. possibilities.
The more options to investigate more fun you will have.

But supose this project is really viable :

In that case the title was missleading : You are in real looking for:
"A binary Forth CROSS-compiler hosted on Linux 64"
=====================================================

You say you can do this on C, or on a Macro-generator
So in essence you describe a CROSS compiler, target is the Pentium Family
of CPUs, you are not interested on an interactive system on your target.
--The output should be pure binary, to be disassembled and studied--


Probably, you will be closer to your goal, starting with a real Forth to binary compiler
like of Tom Almy´s (if you can get the code) or any other similar
f.e. Tcom from Tom.
And adapt the MSdos I/O to Linux, this part is a piece of cake and a well
known road (+documented) .
A good programmer can resolve this part for you in 2 days flat.
If the forth compiler source is in C, you will have no adaption work at all.
===========================================================================

[Anton Ertl]

m...@iae.nl writes:
>On Wednesday, April 15, 2020 at 5:40:01 PM UTC+2, Anton Ertl wrote:
>> Cecil - k5nwa <cba...@cbayona.com> writes:
>> >Why small
>> >? So it can fit in the cache of the CPU and be lightning fast among
>> >other things
>[..]
>> By contrast, --no-super reduces the "branch-misses" (branch
>> mispredictions) by a 172M. If we assume that each misprediction costs
>> 16 cycles (which is realistic), this explains the performance
>> difference of 2.8G cycles.
>
>That is interesting. Is there some special instruction or
>secret sequence that we can use to advance-warn the CPU that
>we are going to branch or call somewhere soon?

Depends on the architecture:

PowerPC has the ctr (counter) and lr (link) registers, and you have to
move the target address to one of these registers (with mtctr or mtlr)
before you branch there (with bctr or blr and its variants); the
implementations could use this as an advance warning; I don't know if
they do so, though. I found that gcc just does not separate them
(probably these registers are not modeled in gcc, and modeling them
would require quite a bit of work).

IA-64 splits indirect branches in a similar way.

IIRC, on Itanium II (or I may be confusing this with the PowerPC 970),
when the branch part is seen, it uses the contents of the register at
that time to predict the target; if the register is set too late, this
results in a mispredicted branch; I dimly remember a latency of 6
cycles that is necessary between the first and the second part.

SPARC acquired some prepare-to-indirect-branch instruction with much
fanfare ("speeds up Java") at some point, but I have not looked
closely at it.

The Intel/AMD side acquired indirect branch predictors relatively
early: 1993 for Intel (Pentium), 1999 for AMD (K7). Initially they
had branch target buffers, later (e.g., Core 2 Duo) they used more
history for predicting indirect branches. This works pretty well and
does not need compiler enhancements that never come.

While CPUs still only had BTBs, we developed replication (--no-super;
and by extension, dynamic superinstructions with replication, the
default in Gforth) to get a better prediction accuracy out of BTBs.

History-based predictors can do already quite well without replication
(for the small benchmarks, Skylake produces the same performance with
--no-dynamic and --no-super; for larger benchmarks, replication still
produces an advantage on Haswell; I have not measured on Skylake). So
I presented results on a Goldmont, where replication still makes a big
difference.

One other aspect is that most high-performance CPUs have a "return
stack" (not the Forth return stack) for predicting returns. This
usually works quite well.

>I suppose that if the hardware knew that we are
>going to jump to $xxxx it could already prefetch
>the code there. A pattern like "pop ebx, jmp ebx,"
>or "ret," etc. is common enough that the hardware
>could speculate with some confidence that the value
>at the top of the return stack is a good candidate
>for an imminent IP change.

It does not do that, and it's hard for an OoO or deeply pipelined
in-order CPU. That's because the front end of the CPU runs far ahead
of the back end where data is handled.

Instead, the CPU's return stack is a separate structure inside the
CPU's front end; when the front end sees a call, it pushes, when it
sees a return, it pulls. If you perform return-address manipulation,
it will mispredict.

>If such a mechanism were in place, it would mean
>that we should avoid using the return stack for data
>because this data effectively hides where we are going to
>jump/return: "pop ebx, pop ecx, ret" would be a bad idea,
>"pop ebx pop ecx ...<useful code>... ret" is then much
>better.

The branch predictor does not see that.

>I guess that RISC V exploits the fact that the
>BL instruction exposes the return address in the link
>register.

AFAIK on RISC-V, the link register is a general-purpose register
(unlike on PowerPC), and the return is an ordinary indirect branch.
Of course, a RISC-V implementation can be designed to work better for
the usual return idiom (with a return stack) than for others.

Some literature:

@InProceedings{ertl&gregg03,
author = "M. Anton Ertl and David Gregg",
title = "Optimizing Indirect Branch Prediction Accuracy in Virtual Machine Interpreters",
crossref = "sigplan03",
OPTpages = "",
url = "http://www.complang.tuwien.ac.at/papers/ertl%26gregg03.ps.gz",
abstract = "Interpreters designed for efficiency execute a huge
number of indirect branches and can spend more than
half of the execution time in indirect branch
mispredictions. Branch target buffers are the best
widely available\mn{on all recent general-purpose
machines?} form of indirect branch prediction;
however, their prediction accuracy for existing
interpretes is only 2\%--50\%. In this paper we
investigate two methods for improving the prediction
accuracy of BTBs for interpreters: replicating
virtual machine (VM) instructions and combining
sequences of VM instructions into superinstructions.
We investigate static (interpreter build-time) and
dynamic (interpreter run-time) variants of these
techniques and compare them and several combinations
of these techniques. These techniques can eliminate
nearly all of the dispatch branch mispredictions,
and have other benefits, resulting in speedups by a
factor of up to 3.17 over efficient threaded-code
interpreters, and speedups by a factor of up to 1.3
over techniques relying on superinstructions alone."
}

@Proceedings{sigplan03,
booktitle = "SIGPLAN Conference on Programming Language
Design and Implementation (PLDI'03)",
title = "SIGPLAN Conference on Programming Language
Design and Implementation (PLDI'03)",
year = "2003",
key = "PLDI '03"
}

@Article{ertl&gregg03jilp,
author = {M. Anton Ertl and David Gregg},
title = {The Structure and Performance of \emph{Efficient}
Interpreters},
journal = {The Journal of Instruction-Level Parallelism},
year = {2003},
volume = {5},
month = nov,
url = {http://www.complang.tuwien.ac.at/papers/ertl%26gregg03jilp.ps.gz},
url2 = {http://www.jilp.org/vol5/v5paper12.pdf},
note = {http://www.jilp.org/vol5/},
abstract = {Interpreters designed for high general-purpose
performance typically perform a large number of
indirect branches (3.2\%--13\% of all executed
instructions in our benchmarks). These branches
consume more than half of the run-time in a number
of configurations we simulated. We evaluate how
accurate various existing and proposed branch
prediction schemes are on a number of interpreters,
how the mispredictions affect the performance of the
interpreters and how two different interpreter
implementation techniques perform with various
branch predictors. We also suggest various ways in
which hardware designers, C compiler writers, and
interpreter writers can improve the performance of
interpreters.}
}

@Article{casey+07toplas,
author = {Kevin Casey and M. Anton Ertl and David Gregg},
title = {Optimizing indirect branch prediction accuracy in
virtual machine interpreters},
journal = toplas,
year = {2007},
volume = {29},
number = {6},
pages = {37:1--37:36},
month = oct,
url = {http://doi.acm.org/10.1145/1286821.1286828},
abstract = {Interpreters designed for efficiency execute a huge
number of indirect branches and can spend more than
half of the execution time in indirect branch
mispredictions. Branch target buffers (BTBs) are the
most widely available form of indirect branch
prediction; however, their prediction accuracy for
existing interpreters is only 2\%--50\%. In this
article we investigate two methods for improving the
prediction accuracy of BTBs for interpreters:
replicating virtual machine (VM) instructions and
combining sequences of VM instructions into
superinstructions. We investigate static
(interpreter build-time) and dynamic (interpreter
runtime) variants of these techniques and compare
them and several combinations of these
techniques. To show their generality, we have
implemented these optimizations in VMs for both Java
and Forth. These techniques can eliminate nearly all
of the dispatch branch mispredictions, and have
other benefits, resulting in speedups by a factor of
up to 4.55 over efficient threaded-code
interpreters, and speedups by a factor of up to 1.34
over techniques relying on dynamic superinstructions
alone.}

[Paul Rubin]

a...@littlepinkcloud.invalid writes:
> Really? I wasn't aware of that. Can't you always do eval and whatnot?
> Or is there a fully-compiled subset with no dictionary in there?

Of course there are subsets that don't have eval. Are they still
standard conformant (Common Lisp)? Maybe not. Aren't there eval-like
things in Forth though? You have to leave them out to get a compileable
subset, but that is fine.

There are also non-CL dialects with no eval, such as Scheme. That's
similar to the situation with Forth's many dialects.

[none albert]

In article <87wo6ge...@nightsong.com>,

Paul Rubin <no.e...@nospam.invalid> wrote:
>a...@littlepinkcloud.invalid writes:
>> Really? I wasn't aware of that. Can't you always do eval and whatnot?
>> Or is there a fully-compiled subset with no dictionary in there?
>
>Of course there are subsets that don't have eval. Are they still
>standard conformant (Common Lisp)? Maybe not. Aren't there eval-like
>things in Forth though? You have to leave them out to get a compileable
>subset, but that is fine.

You mean EVALUATE ?

: EVALUATE SAVE SET-SRC 'INTERPRET CATCH RESTORE THROW ;
(SAVE SET-SRC is a lightweight version of EXECUTE-PARSING)

I don't try to eliminate EVALUATE from my
euler turnkeys ^H^H^H^H^H^H executables because
that saves no space.
With the odd buffer defined executable files are sometimes over 100 Mbyte.

Also EVALUATE comes in handy.

I typically use argument passing for an upper limit:

euler751 1,000,000,000,000,000

or even

euler751 ' "**" WANTED 10 15 ** '

>
>There are also non-CL dialects with no eval, such as Scheme. That's
>similar to the situation with Forth's many dialects.

Groetjes Albert

[Cecil - k5nwa]

> If the forth compiler source is in C, you will have no adaption work at all..
> ===========================================================================
>
>
Unfortunately there is a lot more than modifying the I/O, the code
generator generates 16-bit code which is unusable, to modify it to
generate x64 code is a major undertaking which one might as well write
from scratch.

A very small part is in C, the VM and the primitives are in C the rest
some 140 words are in Forth. Part of the problem is my being ill, in two
days I can accomplish little, dizziness and headaches 24/7 and having
breathing difficulties take their toll and it difficult of getting a lot
done. But like I said some of the systems suggested look to be usable.

Right now I forgot about my taxes and they are due today so I have to
concentrate on that and get it done, after that I can try some of the
suggested software and see how they work out. Two of the suggested
software look promising so I'll start there.

--
Cecil - k5nwa

[Paul Rubin]

albert@cherry.(none) (albert) writes:
> Also EVALUATE comes in handy.
> I typically use argument passing for an upper limit:
> euler751 1,000,000,000,000,000

Gforth 0.7.3 64-bit:

1e15 f>s . 1000000000000000 ok

[Paul Rubin]

Cecil - k5nwa <cba...@cbayona.com> writes:

> Right now I forgot about my taxes and they are due today

There is a 3 month extension due to covid-19.

[Cecil - k5nwa]

Thanks and wow! I missed something by not watching the news beside a lot
of politics and other garbage. That is good so I can take a few days and
have it ready by the weekend, I usually get money back around $1200,
which I will need to replenish my savings.

Right now I have a major headache so I will do some light reading and
maybe do an install some of the software that needs testing. I will
start with :r4 since it's small and simple related to Color Forth but
can generate standalone applications.


--
Cecil - k5nwa

[Paul Rubin]

Cecil - k5nwa <cba...@cbayona.com> writes:
> days and have it ready by the weekend, I usually get money back around
> $1200, which I will need to replenish my savings.

1. If you have a refund coming, then nothing terribly bad happens if you
file late.

2. If you filed last year, you should automatically get $1200 in
coronavirus relief funds separately from whatever tax refund you have
coming, though it's hard to predict exactly when it will come.

[Rick C]

As soon as Trump signs it... He's going to have a sore hand.

--

Rick C.

- Get 1,000 miles of free Supercharging
- Tesla referral code - https://ts.la/richard11209

[hughag...@gmail.com]

On Thursday, April 9, 2020 at 8:46:55 AM UTC-7, Cecil - k5nwa wrote:
> I want to work on a special educational project so I'm looking for a
> Forth with special qualities. I will use this software to build other
> Forth Systems with optimizing capabilities.
>
> It needs to be able to generate a stand alone target program that has a
> minimum code in it so it should not include the Forth System in it just
> the desired code. So in essence it's a target compiler for the Pentium
> Family of CPUs.

If you want a cross-compiler, I know how to do that.

Normally a cross-compiler is for a target processor that doesn't
support a traditional interactive Forth system. My first cross-compiler
was for the 65c02 and it allowed me to write a gigantic application
program that would have been impossible with a traditional Forth.
My second cross-compiler was for the MiniForth at Testra. This was
necessary because code-memory wasn't writable at run-time, so it was
impossible to have a traditional Forth with an assembler. After I left,
a traditional Forth was written that required all the primitives to
already be provided by my MFX. This was interactive so it was easier
to work with than MFX, making program development faster assuming that
performance requirements were minimal so assembly programming wasn't needed.

Cross-compilation was done for the 16-bit x86 in the old days when the
16-bit x86 was used as a micro-controller (typically the 80186
or the V30 etc.). This was a bad idea because the x86 has a huge
interrupt latency and hence makes a terrible micro-controller.
This was mostly done for systems that had very minimal performance
requirements, and the programmers wanted an easy assembly-language
that they already knew.

I have no idea why you would want to do anything like this now.
You said that it is "educational." What does that mean?

I would expect that most Forth systems have a "turn-key" option
that allows you to generate an executable that runs some function
other than the interpreter. The executable may or may not include
the dictionary and interpreter, which may or may not matter to you.
If it doesn't have this, then you generate an application program,
and the user doesn't know it is written in Forth (and likely doesn't care).
If it does have this, then your application program has Forth
as a scripting language for the user to work with (similar to how
some C programs have Lua as a scripting language for the user).
I would expect that most users would prefer Lua to Forth these days.

If you told me what you are trying to accomplish, I might be able to help.
Without any idea of where you want to go, nobody is going to
point you in the right direction.

I hope your health improves.
Look into what I said about mold. Working as a plumber, I can tell you
that a leak may not be visible, but you can have mold growing that
produces spores that cause the symptoms you describe. A plumbing leak
can cause mold inside of the walls or under the floor. A roof leak
can cause mold in the attic, which may get into the A/C ducts.
The primary symptom is that when you breath, the air feels "heavy."
There are instruments that test the air for mold spores,
but I don't know anything about how that is done, or who does this.
Mold is not dangerous when it is wet. When it dries though, it produces
spores. This looks like white dust on top of the black mold. This is
very dangerous. If you see this, run! Fixing this problem requires
a special license --- the average plumber can't do mold alleviation
(this actually gets done all the time, but it is not legal).

Try moving out of your house. Go on a vacation. If this helps,
then you can be pretty sure that your house is the cause of your problems.
This may be mold, but there are other possibilities. If your house was
built on top of a landfill, there could be chemicals seeping through
the basement walls that give off poisonous fumes.
If you can't afford a vacation, try sleeping in a hammock outside,
and only go inside for chores such as showering and cooking, but stay
outside as much as possible --- it is springtime, so the weather is nice.
Fresh air never hurt anybody, and it might help. :-)

[Cecil - k5nwa]

Yes I would like to get a cross-compiler that generates x64 code and
runs in Linux or can run in Wine. I want to do some experiments with
code optimization so I need to easily be able to easily change the
generated code as I carry out experiments.

I don't know of any, the several Forth I'm aware of like Vfx and
SwiftForth carry the whole compiler along with the application, I rather
not do that as I want to analyze the generated code and if 90% is
something else not having to do with the application it makes things
harder.

Vfx has a target compiler but I have not found any documentation on it
and besides its optimizing code generator is quite complex and not
easily changed.

SwiftForth is generally less complex so it might be easier to figure it
out in some cases it generates not as efficient code but that is OK, a
big issue with these two compilers is they generate i386 code instead of
x64 code, and it would be a huge task to change that.

In the past when I did experiments like this I wrote the compiler and
optimizer in C, I rather not do it like that but it did work well and I
remember enough to be able to do it if I had no choice but I rather do
it in Forth if possible and better yet find a Forth that does some of it
itself.

At that time there was a text file containing the rules on how to
generate code and how to optimize the code, it was very easy to change
the rules by editing the "rules file" and no changes to the application
was needed. The rules were stored in a character based doubly linked
list, inefficient regarding space but efficient at finding complex
pattern matches.

It looks more and more that I will either have to write the whole thing
in Forth from scratch or use some other language like C, Prolog,
Perl6,or Lisp to do it, C and Prolog I can do, Lisp and Perl6 I'm not
familiar with them. Part of the project involves massive text
manipulation an area that is very weak in Forth but Prolog, Perl6, and
Lisp can tackle with less effort, even C is not bad.
_______________________________________

As far as health issues, and more details I will send you a private
email tomorrow, it's late here.



--
Cecil - k5nwa

[dxforth]

On Wednesday, April 15, 2020 at 11:34:45 PM UTC+10, a...@littlepinkcloud.invalid wrote:
> dxforth <dxf...@gmail.com> wrote:
>
> > "Reverse-Polish C" is an accurate description of ANS - the inevitable
> > consequence of standardization, portability and locals.
>
> Not really, no: the strengths and weaknesses of the languages are very
> different. It's hard for me to find many similarities beyond the
> inevitable ones that come from being standardized languages that try
> to provide low-level access.

How different when the goals and solutions are the same as C's.
Being a success story, C has become Forth's role model. That
its own users have come to see Forth as immature and lacking is
apparent everywhere - from calls to standardize strings, to a TC
that deems it necessary to sit in perpetuity.

>
> > It would prompt Jeff Fox to say:
> >
> > "I see ANSI Forth as [...] one more failed attempt to find the ideal
> > portable scripting language."
>
> Jeff Fox said many things, some of them unwise.
>
> Andrew.

Unwise only if something had changed in the two decades since it was written.

[dxforth]

On Thursday, April 16, 2020 at 1:35:36 PM UTC+10, Rick C wrote:
> ...

> As soon as Trump signs it... He's going to have a sore hand.

He gets that from tweeting - then having to walk back from it a day later :)

[a...@littlepinkcloud.invalid]

dxforth <dxf...@gmail.com> wrote:
> On Wednesday, April 15, 2020 at 11:34:45 PM UTC+10, a...@littlepinkcloud.invalid wrote:
>> dxforth <dxf...@gmail.com> wrote:
>>
>> > "Reverse-Polish C" is an accurate description of ANS - the inevitable
>> > consequence of standardization, portability and locals.
>>
>> Not really, no: the strengths and weaknesses of the languages are very
>> different. It's hard for me to find many similarities beyond the
>> inevitable ones that come from being standardized languages that try
>> to provide low-level access.
>
> How different when the goals and solutions are the same as C's.
> Being a success story, C has become Forth's role model.
> That its own users have come to see Forth as immature and lacking is
> apparent everywhere - from calls to standardize strings, to a TC
> that deems it necessary to sit in perpetuity.

Just look at this paragraph for a moment. As far as I can tell it's
entirely composed of opinions stated as facts, with no supporting
evidence or even any discernible connection with reality.

>> > It would prompt Jeff Fox to say:
>> >
>> > "I see ANSI Forth as [...] one more failed attempt to find the ideal
>> > portable scripting language."
>>
>> Jeff Fox said many things, some of them unwise.
>

> Unwise only if something had changed in the two decades since it was
> written.

What does that even mean?

Andrew.

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>There have always been two distinct ways of thinking about Forth, and
>>writing Forth programs. One school tends to compile the Forth program,
>>then run it, without any runtime use of the dictionary and the
>>interpreter. (I've sometimes mischeivously referred to this as
>>"Reverse-Polish C.") The other recognizes that Forth is at heart a
>>dynamic programming language capable of runtime code generation,
>>interpretation, and adaptation, and uses all these tools in a sort-of
>>LISPy way. The first, of course, is the only recourse available to
>>anyone generating entirely-statically-compiled binaries, especially
>>for embedded systems. It's very cool that Forth is flexible enough to
>>accomodate both of these usages; I don't know of any other language
>>that can so comfortably do that.
>
> Not sure about "so comfortably", but getting rid of the text
> interpreter/REPL/etc. seems to be very common for languages that have
> one (at least for older ones from a time when memory was still
> scarce). Various BASIC compilers (without REPL)

Hmm, but I'm talking about programming practice here. I know that some
BASICs had the equivalent of EVALUATE but was it ever a common idiom?
I saw a few times people would demonstrate it as a kind of party
trick, but that's not really the same thing; or at least it's not the
thing I'm talking about, which is static-versus-dynamic programming
style.

IIRC early Microsoft BASICs didn't even have any kind of EVALUATE, but
I remember people pushing stuff into the input buffer, then exiting
from a program; the system would then interpret the stuff pushed into
the bugger and then restart the program. Worked but was fugly.

> and Borland Prolog (which was also a big deviation from mainstream
> Prolog in other aspects) come to my mind.

OK. I don't know enough about Prolog to comment.

Andrew.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> a...@littlepinkcloud.invalid writes:
>>>There have always been two distinct ways of thinking about Forth, and
>>>writing Forth programs. One school tends to compile the Forth program,
>>>then run it, without any runtime use of the dictionary and the
>>>interpreter. (I've sometimes mischeivously referred to this as
>>>"Reverse-Polish C.") The other recognizes that Forth is at heart a
>>>dynamic programming language capable of runtime code generation,
>>>interpretation, and adaptation, and uses all these tools in a sort-of
>>>LISPy way. The first, of course, is the only recourse available to
>>>anyone generating entirely-statically-compiled binaries, especially
>>>for embedded systems. It's very cool that Forth is flexible enough to
>>>accomodate both of these usages; I don't know of any other language
>>>that can so comfortably do that.
>>
>> Not sure about "so comfortably", but getting rid of the text
>> interpreter/REPL/etc. seems to be very common for languages that have
>> one (at least for older ones from a time when memory was still
>> scarce). Various BASIC compilers (without REPL)
>
>Hmm, but I'm talking about programming practice here. I know that some
>BASICs had the equivalent of EVALUATE but was it ever a common idiom?

I don't know. Microsoft BASIC on the C64 did not have it. Anyway,
it's not a common idiom in Forth. I did not mean EVALUATE. I meant a
text interpreter/REPL. In C64 BASIC I can do stuff on the command
line

print a
b$="bla"
goto 25

I cannot do that with compiled REPL-less code; the compiled code does
not know the names of variables, or line numbers.

>I saw a few times people would demonstrate it as a kind of party
>trick, but that's not really the same thing; or at least it's not the
>thing I'm talking about, which is static-versus-dynamic programming
>style.

You are speicifically talking about EVALUATE. While a system with a
text interpreter can add EVALUATE without much effort, IMO neither the
presence nor a significant usage frequency of EVALUATE are necessary
to make a language capable of run-time code generation and
interpretation (not sure what you mean with adaption).

Conversely, the presence of EVALUATE does not indicate much about
dynamic abilities. E.g., Forth-94 has EVALUATE, but it has no nts,
and xts are limited to being EXECUTEd and COMPILE,d. There is no way
to get interpretation semantics, compilation semantics, or the name of
a word from the xt. Fortunately, Forth-2012 adds these capabilities
to standard Forth.

Of course, that is not exhaustive. Additional meta-information would
be useful, and some of it is present in various systems, and is used
by programmers in system-specific ways. It is one of the aspects of
Forth that attracted me to the language. In the meantime other
languages have overtaken Forth in that aspect.

[A. K.]

The thing with modern CPUs is that they vary A LOT from one generation to
the next. You optimize for one machine and most of all this tedious work
is lost within short time. Perhaps even worse for programmers are instruction
set developments. That's why some C compiler vendors offer intrinsics instead
of a full-fledged assembler. Intrinsics hide the implemtation details while
still offering low-level CPU access. It's a small abstraction layer that
can be maintained through different CPU generations. Intel's icc compiler
generates efficient code because it contains waggon-loads of conditional
compilations depending on the detected CPU.

That considered, if I were up to optimzation and pattern matching eg by holding
Forth code in Prolog clauses, I would not endeavour to go down to assembler
level - but that's just me. Instead I would select or build a good virtual
machine as abstraction layer. The VM instruction set will not change much over
time and provide a stable founding for optimizing the Forth running on it.

[a...@littlepinkcloud.invalid]

>>I saw a few times people would demonstrate it as a kind of party
>>trick, but that's not really the same thing; or at least it's not the
>>thing I'm talking about, which is static-versus-dynamic programming
>>style.
>
> You are speicifically talking about EVALUATE.

Not exclusively. As I said my first pose, "a dynamic programming

language capable of runtime code generation, interpretation, and

adaptation." If you're going to do any kind of runtime code generation
or interpretation in BASIC, some kind of EVALUATE is the only tool in
the box that could do that, AFAIAA. In Forth you have many tools,
including CREATE and : . But in BASIC some kind of EVALUATE is all
you've got.

> While a system with a text interpreter can add EVALUATE without much
> effort, IMO neither the presence nor a significant usage frequency
> of EVALUATE are necessary to make a language capable of run-time
> code generation and interpretation

Certainly not, no.

> (not sure what you mean with adaption).

Changing code on the fly, in response to events.

Clear now?

Andrew.

[Anton Ertl]

"A. K." <minf...@arcor.de> writes:
>The thing with modern CPUs is that they vary A LOT from one generation to
>the next. You optimize for one machine and most of all this tedious work
>is lost within short time.

I am not sure what kind of optimization you have in mind, but for the
level of things I work at, the CPU generations do not vary much. Yes,
Haswell has a better indirect branch predictor than Sandy Bridge, so
some of our optimizations are aleady subsumed by the newer CPU (at
least for small programs), but overall, what worked well on a K8 from
2003 or even a Coppermine from 1999 still works well now.

Conversely, writing close to executable code was already slow on the
Pentium in 1993, and it is still slow today. Keeping the loop index
in memory has always been slow, but this effect has become more severe
over time: these days, it means that a loop iteration takes at least 5
cycles, and the CPU can perform about 20 instruction in these 5 cycles.

BTW, avoiding slow idioms, like the two mentioned above, seems more
important to me (and has become more important over time) than finding
some kind of perfect idiom for some other part.

>Perhaps even worse for programmers are instruction
>set developments. That's why some C compiler vendors offer intrinsics instead
>of a full-fledged assembler. Intrinsics hide the implemtation details while
>still offering low-level CPU access. It's a small abstraction layer that
>can be maintained through different CPU generations.

The main benefits of intrinsics over writing assembly language is that
the compiler performs register allocation for you, and you can mix it
with C code.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>As I said my first pose, "a dynamic programming
>language capable of runtime code generation, interpretation, and
>adaptation." If you're going to do any kind of runtime code generation
>or interpretation in BASIC, some kind of EVALUATE is the only tool in
>the box that could do that, AFAIAA.

I see.

>In Forth you have many tools,
>including CREATE and : .

Both of which require parameters from the input stream, which severly
limits their usefulness in meta-programming.

>> (not sure what you mean with adaption).
>
>Changing code on the fly, in response to events.

Standard Forth does not have anything more than, e.g., C, in this
area, and I am not sure it should.

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>

>>In Forth you have many tools, including CREATE and : .
>
> Both of which require parameters from the input stream, which
> severly limits their usefulness in meta-programming.

It's an inconvenience, surely. Didn't you write EXECUTE-PARSING in
Standard Forth?

>>> (not sure what you mean with adaption).
>>
>>Changing code on the fly, in response to events.
>
> Standard Forth does not have anything more than, e.g., C, in this
> area, and I am not sure it should.

In Standard Forth you can FORGET stuff, reload, and restart. In what
way does that not qualify? You sure as hell can't do that in C.

Andrew.

[Paul Rubin]

a...@littlepinkcloud.invalid writes:
> It's hard for me to find many similarities beyond the inevitable ones
> that come from being standardized languages that try to provide
> low-level access.

I wonder whether there's really that much point to standardized
languages that try to provide low-level access these days. I think low
level access basically means machine pointers as a native datatype in
the language. An alternative might be to just have a garden variety
Algol-like language with a way to call assembly language subroutines.

[dxforth]

On Thursday, April 16, 2020 at 6:00:37 PM UTC+10, a...@littlepinkcloud.invalid wrote:
> dxforth <dxf...@gmail.com> wrote:
> > On Wednesday, April 15, 2020 at 11:34:45 PM UTC+10, a...@littlepinkcloud.invalid wrote:
> >> dxforth <dxf...@gmail.com> wrote:
> >>
> >> > "Reverse-Polish C" is an accurate description of ANS - the inevitable
> >> > consequence of standardization, portability and locals.
> >>
> >> Not really, no: the strengths and weaknesses of the languages are very
> >> different. It's hard for me to find many similarities beyond the
> >> inevitable ones that come from being standardized languages that try
> >> to provide low-level access.
> >
> > How different when the goals and solutions are the same as C's.
> > Being a success story, C has become Forth's role model.
> > That its own users have come to see Forth as immature and lacking is
> > apparent everywhere - from calls to standardize strings, to a TC
> > that deems it necessary to sit in perpetuity.
>
> Just look at this paragraph for a moment. As far as I can tell it's
> entirely composed of opinions stated as facts, with no supporting
> evidence or even any discernible connection with reality.

I don't need to have supporting evidence or connection with reality.
Unfortunately one can't say the same of Forth.

[dxforth]

IMO Forth parallels assembler in that respect. In the CP/M PD era there
were folks who had written substantial 'standard' asm libraries. They
were free, with source, great quality, readily available and downloaded.
Yet hardly anyone appears to have used them. I put that down to the
low-level nature of asm. Programmers dealing with code at that level
would rather write their own. The higher the abstraction in a language,
the less fussy programmers are about what underlies it, and the more
accommodating they are to using what they've been given.

[m...@iae.nl]

On Thursday, April 16, 2020 at 4:19:52 PM UTC+2, A. K. wrote:
[..]

> The thing with modern CPUs is that they vary A LOT from one generation
> to the next. You optimize for one machine and most of all this tedious
> work is lost within short time.

That is not my experience at all! I had to *add* a rule for combined
code/data space, and Anton makes a good case for maybe two others.

Do your remember any examples where an old rule didn't work anymore?

-marcel

[none albert]

In article <dcaad6e6-d4a2-4cec...@googlegroups.com>,

The first rules of optimisation are
- get rid of superfluous memory accesses
- get rid of superfluous jumps
- get rid of superfluous instructions

This works accross the board from the 8086 to the AMD.

Try to single step through a system call on MS-Windows.
The amount of jumping around, copying and moving of stuff
before some real work is done is just incredible.
Difference between CPUs are at the very bottom of speed
considerations.

>
>-marcel

[A. K.]

I was not referring to backwards compatibility. Of course one can still
program like in the 80's by reducing oneself to a minimal x86 instruction
set and 5 or so CPU registers. If this is okay, fine, because it makes
life easy.

But to squeeze more horsepower out of modern CPUs, one should not suppress
64-bit register widths, considerably more registers, 80-bit fp-numbers,
extended vector instruction sets etc. etc. Caching and branch misprediction
considerations notwithstandung.

So IMO one has to think twice and strike a good balance between simplicity
and performance maximum in the optimization. I am in for even more simplicity
by suggesting to use a VM.

[a...@littlepinkcloud.invalid]

Paul Rubin <no.e...@nospam.invalid> wrote:
> a...@littlepinkcloud.invalid writes:
>> It's hard for me to find many similarities beyond the inevitable ones
>> that come from being standardized languages that try to provide
>> low-level access.
>
> I wonder whether there's really that much point to standardized
> languages that try to provide low-level access these days. I think
> low level access basically means machine pointers as a native
> datatype in the language.

Pretty much, yes. There's also a big chunk of code that does stuff
like create stack frames, suspend and resume tasks, and so on. There's
never been a nice way to write that in any HLL, from what I've seen.

Another important challenge is writing a garbage collector: you really
do need to be able to do things which break every "rule" a HLL might
want to impose. Scanning the stacks for roots, for example.

> An alternative might be to just have a garden variety Algol-like
> language with a way to call assembly language subroutines.

Maybe, but it's a hell of a lot easier to write, say, a device driver
in a high level language if you can.

Andrew.

[Anton Ertl]

Paul Rubin <no.e...@nospam.invalid> writes:
>a...@littlepinkcloud.invalid writes:
>> It's hard for me to find many similarities beyond the inevitable ones
>> that come from being standardized languages that try to provide
>> low-level access.
>
>I wonder whether there's really that much point to standardized
>languages that try to provide low-level access these days.

Concerning low-level languages: A while ago I listened to a number of
talks on security. One of the presenters said that 90% of the CPU
cycles are spent in C code (or at least that's how I interpreted him).
He used that as justification for investigating various mitigation
techniques for security problems in C code.

But it also points to a need for low-level languages. Why have, e.g.,
the JPEG decoding libraries not been rewritten in Java when security
holes were found? I suspect that such libraries would probably be
quite a bit slower if written in Java.

Not sure about the "standardized" part.

>I think low
>level access basically means machine pointers as a native datatype in
>the language.

That's typical, but not exhaustive.

>An alternative might be to just have a garden variety
>Algol-like language with a way to call assembly language subroutines.

Java and JNI? Now what? Write the JPEG library in the assembly
languages of a bunch of architectures? What would that win?

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> a...@littlepinkcloud.invalid writes:
>>>
>>>In Forth you have many tools, including CREATE and : .
>>
>> Both of which require parameters from the input stream, which
>> severly limits their usefulness in meta-programming.
>
>It's an inconvenience, surely. Didn't you write EXECUTE-PARSING in
>Standard Forth?

Yes, it's possible, barely (not on systems without ALLOCATE). The
fact that I had to write it, instead of it being present in Forth
systems (and it's still not in SwiftForth, VFX, nor iForth; it exists
in lxf and Gforth) and that nobody has proposed it for standardization
(and the more specialized :NAME has been an RfD without CfV for >10
years) demonstrates that standard Forth is not a language for
meta-programmers.

>>>> (not sure what you mean with adaption).
>>>
>>>Changing code on the fly, in response to events.
>>
>> Standard Forth does not have anything more than, e.g., C, in this
>> area, and I am not sure it should.
>
>In Standard Forth you can FORGET stuff, reload, and restart. In what
>way does that not qualify?

It's easier to use deferred words (which is equivalent to what C
offers), so why would anybody do that? And indeed, AFAIK nobody has
ever done that, with the following exceptions:

1) During development, to load code that was manually changed, not in
response to some non-developer event.

2) I have read about it being used to implement overlays. If you have
enough memory (or demand-paging), you do not do that. So this is not
"changing code on the fly, in response to events".

> You sure as hell can't do that in C.

dlopen()/dlclose().

[none albert]

In article <2020Apr1...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>a...@littlepinkcloud.invalid writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> a...@littlepinkcloud.invalid writes:
>>>>
>>>>In Forth you have many tools, including CREATE and : .
>>>
>>> Both of which require parameters from the input stream, which
>>> severly limits their usefulness in meta-programming.
>>
>>It's an inconvenience, surely. Didn't you write EXECUTE-PARSING in
>>Standard Forth?
>
>Yes, it's possible, barely (not on systems without ALLOCATE). The
>fact that I had to write it, instead of it being present in Forth
>systems (and it's still not in SwiftForth, VFX, nor iForth; it exists
>in lxf and Gforth) and that nobody has proposed it for standardization
>(and the more specialized :NAME has been an RfD without CfV for >10
>years) demonstrates that standard Forth is not a language for
>meta-programmers.

As soon as I heard about it I added it to the ciforth library
in order to be able to run Gforth programs:

76 LIST
..
3 : EXECUTE-PARSING ROT ROT SAVE SET-SRC CATCH RESTORE THROW ;
..

The smallness of this definition suggests that the facility
was in practice present in ciforth.
QUIT and EVALUATE use the same words: SAVE RESTORE SET-SRC

>
>>>>> (not sure what you mean with adaption).
>>>>
>>>>Changing code on the fly, in response to events.
>>>
>>> Standard Forth does not have anything more than, e.g., C, in this
>>> area, and I am not sure it should.
>>
>>In Standard Forth you can FORGET stuff, reload, and restart. In what
>>way does that not qualify?
>
>It's easier to use deferred words (which is equivalent to what C
>offers), so why would anybody do that? And indeed, AFAIK nobody has
>ever done that, with the following exceptions:

I remember a lecture where there was cluster of Forth's that
controlled a nuclear plant that could never shut down.
That could be an example where code was adapted on the fly.
The same for the Forth's that sits in same craft.
>SNIP>
>- anton

[Howerd]

On Friday, 17 April 2020 16:43:31 UTC+2, Anton Ertl wrote:
> aph.....pinkcloud.invalid writes:
> >Anton Ertl <anton....s.complang.tuwien.ac.at> wrote:

Hi Anton,

In colorForth, Chuck defines
: dump ( a -- ) 32 load ;

where block 32 contains the red definition on dump plus the yellow interpreted execution of dump. The app that you are developing starts with empty , but block 32 does not. It all hangs together quite nicely.

The system blocks are loaded into memory at power up, so block is defined as
: block ( n -- a ) $400 * MEM_BASE + ;

Compilation is very fast, because it just has to search a 256 element list using a single machine instruction repne scasd .

There is no need for Wordlists etc, you just empty and start again...
Its a different way of doing things :-)

And yes, ancient polyForths with two 720K floppy drives used to fetch pre-compiled overlays from disk. I wrote a program once where each Function key pulled in its own overlay, so I could get one of ten 64K sub-programs loading in a relatively quick time. When this is moved onto a hard disk the delay is not noticeable.

Cheers,
Howerd

[m...@iae.nl]

On Friday, April 17, 2020 at 4:43:31 PM UTC+2, Anton Ertl wrote:
> a...@littlepinkcloud.invalid writes:
> >Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> >> a...@littlepinkcloud.invalid writes:
> >>>
> >>>In Forth you have many tools, including CREATE and : .
> >>
> >> Both of which require parameters from the input stream, which
> >> severly limits their usefulness in meta-programming.
> >
> >It's an inconvenience, surely. Didn't you write EXECUTE-PARSING in
> >Standard Forth?
>
> Yes, it's possible, barely (not on systems without ALLOCATE). The
> fact that I had to write it, instead of it being present in Forth
> systems (and it's still not in SwiftForth, VFX, nor iForth; it exists
> in lxf and Gforth) and that nobody has proposed it for standardization
> (and the more specialized :NAME has been an RfD without CfV for >10
> years) demonstrates that standard Forth is not a language for
> meta-programmers.

FORTH> help $PROCESS
$PROCESS "string-process" IFORTH
( c-addr u xt -- )
The counted string described by c-addr and u becomes the temporary
input buffer during the execution of xt. The following code is
equivalent:

CREATE foo 4 ,
S" foo " ' CREATE $PROCESS 4 ,

See also: EVALUATE
ok

-marcel

[m...@iae.nl]

On Friday, April 17, 2020 at 1:55:46 PM UTC+2, A. K. wrote:
> Am Freitag, 17. April 2020 13:05:46 UTC+2 schrieb m...@iae.nl:
> > On Thursday, April 16, 2020 at 4:19:52 PM UTC+2, A. K. wrote:
> > [..]
> > > The thing with modern CPUs is that they vary A LOT from one generation
> > > to the next. You optimize for one machine and most of all this tedious
> > > work is lost within short time.
> >
> > That is not my experience at all! I had to *add* a rule for combined
> > code/data space, and Anton makes a good case for maybe two others.
> >
> > Do your remember any examples where an old rule didn't work anymore?
> >
>
> I was not referring to backwards compatibility. Of course one can still
> program like in the 80's by reducing oneself to a minimal x86 instruction
> set and 5 or so CPU registers. If this is okay, fine, because it makes
> life easy.

One of us two is not understanding the subject of this thread.
I was referring to Anton's post.

-marcel

[A. K.]

I was referring to Cecil who started and defined the subject of this thread.

[hughag...@gmail.com]

On Friday, April 17, 2020 at 7:43:31 AM UTC-7, Anton Ertl wrote:
> a...@littlepinkcloud.invalid writes:

> Didn't you write EXECUTE-PARSING in
> >Standard Forth?
>
> Yes, it's possible, barely (not on systems without ALLOCATE). The
> fact that I had to write it, instead of it being present in Forth
> systems (and it's still not in SwiftForth, VFX, nor iForth; it exists
> in lxf and Gforth) and that nobody has proposed it for standardization
> (and the more specialized :NAME has been an RfD without CfV for >10
> years) demonstrates that standard Forth is not a language for
> meta-programmers.

I wrote :NAME using my <CSTR circular buffer rather than ALLOCATE
and there is no reason why EXECUTE-PARSING couldn't also be written
to use the <CSTR circular buffer. My STRING-STACK.4TH would also work,
but it uses ALLOCATE internally. Some of the crude string-stacks
don't use ALLOCATE at all, but I wouldn't use them for anything.
I like :NAME and rely heavily on it throughout the novice-package.

I agree that ANS-Forth is not a language for meta-programmers.
Everything is needlessly difficult. The lack of a control-flow stack
is an obvious problem. There are multiple other problems.
I don't think Elizabeth Rather knew what meta-programming is,
because this is beyond the scope of the "Starting Forth" book.

[hughag...@gmail.com]

On Thursday, April 16, 2020 at 12:41:33 AM UTC-7, dxforth wrote:
> Being a success story, C has become Forth's role model.

This is really not true!

DXForth says "Forth's" as if Forth were a person.
This isn't true though. DXForth is actually referring to the
Forth-200x clown committee who claim to represent Forth,
but this is not "Forth" in any meaningful way.

My rquotations are far beyond anything that ANSI-C has
(although roughly comparable to what the non-standard GCC has).
The Forth-200x committee is absolutely opposed to the rquotations
(Anton Ertl and Bernd Paysan blatantly lied about the rquotations
in their EuroForth-2018 paper). This is because they are afraid of
providing Forth with features that C lacks. They are just C programmers.

[Anton Ertl]

m...@iae.nl writes:
>FORTH> help $PROCESS
>$PROCESS "string-process" IFORTH
> ( c-addr u xt -- )
> The counted string described by c-addr and u becomes the temporary
> input buffer during the execution of xt. The following code is
> equivalent:
>
> CREATE foo 4 ,
> S" foo " ' CREATE $PROCESS 4 ,

Great! Even the same stack effect.

[Anton Ertl]

albert@cherry.(none) (albert) writes:
>As soon as I heard about it I added it to the ciforth library
>in order to be able to run Gforth programs:
>
>76 LIST
>..
> 3 : EXECUTE-PARSING ROT ROT SAVE SET-SRC CATCH RESTORE THROW ;
>..
>
>The smallness of this definition suggests that the facility
>was in practice present in ciforth.
>QUIT and EVALUATE use the same words: SAVE RESTORE SET-SRC

EXECUTE-PARSING is a more general relative of EVALUATE, so if a system
has EVALUATE, it can also implement EXECUTE-PARSING with the same
components; in Gforth, EVALUATE is built on top of EXECUTE-PARSING:

: evaluate ( ... addr u -- ... ) \ core,block
['] interpret2 execute-parsing ;

[Stephen Pelc]

On Fri, 17 Apr 2020 09:23:08 -0700 (PDT), m...@iae.nl wrote:

>$PROCESS "string-process" IFORTH
> ( c-addr u xt -- )
> The counted string described by c-addr and u becomes the temporary
> input buffer during the execution of xt. The following code is
> equivalent:
>
> CREATE foo 4 ,
> S" foo " ' CREATE $PROCESS 4 ,
>
> See also: EVALUATE
> ok

What do you use this word for? What triggered its introduction?

I ask because if $PROCESS is the same as EXECUTE-PARSING I find the
description much more comprehensible. EXECUTE-PARSING has been
suggested for standardisation but many people found the name
unacceptable. Other candidates could be PARSES or PARSE-WITH.

Stephen

--
Stephen Pelc, ste...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612
web: http://www.mpeforth.com - free VFX Forth downloads

[Anton Ertl]

ste...@mpeforth.com (Stephen Pelc) writes:
>On Fri, 17 Apr 2020 09:23:08 -0700 (PDT), m...@iae.nl wrote:
>
>>$PROCESS "string-process" IFORTH
>> ( c-addr u xt -- )
>> The counted string described by c-addr and u becomes the temporary
>> input buffer during the execution of xt. The following code is
>> equivalent:
>>
>> CREATE foo 4 ,
>> S" foo " ' CREATE $PROCESS 4 ,
>>
>> See also: EVALUATE
>> ok
>
>What do you use this word for? What triggered its introduction?
>
>I ask because if $PROCESS is the same as EXECUTE-PARSING I find the
>description much more comprehensible.

What makes

| Make addr u the current input source, execute xt '( ... -- ... )',
|then restore the previous input source.

much less comprehensible IYO?

[none albert]

In article <5e9af0d9....@news.eternal-september.org>,

Stephen Pelc <ste...@mpeforth.com> wrote:
>On Fri, 17 Apr 2020 09:23:08 -0700 (PDT), m...@iae.nl wrote:
>
>>$PROCESS "string-process" IFORTH
>> ( c-addr u xt -- )
>> The counted string described by c-addr and u becomes the temporary
>> input buffer during the execution of xt. The following code is
>> equivalent:
>>
>> CREATE foo 4 ,
>> S" foo " ' CREATE $PROCESS 4 ,
>>
>> See also: EVALUATE
>> ok
>
>What do you use this word for? What triggered its introduction?
>
>I ask because if $PROCESS is the same as EXECUTE-PARSING I find the
>description much more comprehensible. EXECUTE-PARSING has been
>suggested for standardisation but many people found the name
>unacceptable. Other candidates could be PARSES or PARSE-WITH.

The same here. Each time I encountered EXECUTE-PARSING I had to
look long and hard what it was meant to mean.
I never would have realized that Marcel example could be done
with EXECUTE-PARSING. (!)

>
>Stephen

[Anton Ertl]

albert@cherry.(none) (albert) writes:
>In article <5e9af0d9....@news.eternal-september.org>,
>Stephen Pelc <ste...@mpeforth.com> wrote:
>>On Fri, 17 Apr 2020 09:23:08 -0700 (PDT), m...@iae.nl wrote:
>>> S" foo " ' CREATE $PROCESS 4 ,

...

>I never would have realized that Marcel example could be done
>with EXECUTE-PARSING. (!)

That's surprising; searching for "create execute-parsing" among my
postings, I find this sequence in

<2004Aug3...@mips.complang.tuwien.ac.at>
<2004Aug3...@mips.complang.tuwien.ac.at>
<2006Feb1...@mips.complang.tuwien.ac.at>
<2007Feb1...@mips.complang.tuwien.ac.at>
<n62un1$aav$1...@dont-email.me> (cited by me)
<2016Mar2...@mips.complang.tuwien.ac.at>
<2016May2...@mips.complang.tuwien.ac.at>
<2016Sep2...@mips.complang.tuwien.ac.at>

[dxforth]

On Sunday, April 19, 2020 at 2:29:49 AM UTC+10, none albert wrote:
> In article <5e9af0d9....@news.eternal-september.org>,
> Stephen Pelc <ste...@mpeforth.com> wrote:
> >On Fri, 17 Apr 2020 09:23:08 -0700 (PDT), m...@iae.nl wrote:
> >
> >>$PROCESS "string-process" IFORTH
> >> ( c-addr u xt -- )
> >> The counted string described by c-addr and u becomes the temporary
> >> input buffer during the execution of xt. The following code is
> >> equivalent:
> >>
> >> CREATE foo 4 ,
> >> S" foo " ' CREATE $PROCESS 4 ,
> >>
> >> See also: EVALUATE
> >> ok
> >
> >What do you use this word for? What triggered its introduction?
> >
> >I ask because if $PROCESS is the same as EXECUTE-PARSING I find the
> >description much more comprehensible. EXECUTE-PARSING has been
> >suggested for standardisation but many people found the name
> >unacceptable. Other candidates could be PARSES or PARSE-WITH.
>
> The same here. Each time I encountered EXECUTE-PARSING I had to
> look long and hard what it was meant to mean.
> I never would have realized that Marcel example could be done
> with EXECUTE-PARSING. (!)

In my Miscellaneous definitions:

\ Feed string a u to parsing word xt
sys @ system
: FEED ( adr len xt -- )
-rot s" EXECUTE " >pad +string evaluate ;
sys !

Named after a related function in LMI Forth. Can't say I've
used it though.

[Ruvim]

On 2020-04-17 17:07, Anton Ertl wrote:
> a...@littlepinkcloud.invalid writes:
>> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> a...@littlepinkcloud.invalid writes:
>>>>
>>>> In Forth you have many tools, including CREATE and : .
>>>
>>> Both of which require parameters from the input stream, which
>>> severly limits their usefulness in meta-programming.
>>
>> It's an inconvenience, surely. Didn't you write EXECUTE-PARSING in
>> Standard Forth?
>
> Yes, it's possible, barely (not on systems without ALLOCATE). The
> fact that I had to write it, instead of it being present in Forth
> systems (and it's still not in SwiftForth, VFX, nor iForth; it exists
> in lxf and Gforth) and that nobody has proposed it for standardization
> (and the more specialized :NAME has been an RfD without CfV for >10
> years) demonstrates that standard Forth is not a language for
> meta-programmers.

The same word under EVALUATE-WITH name [1] is present in SP-Forth/4
since its first version in 2001.

It was used (and intended) mainly for parsing arbitrary strings using
words like PARSE, PARSE-NAME, etc. Using this word to pass an argument
via the input stream is just a workaround for absence of a standard
postfix API.

OTOH, I don't think we need to standardize a postfix variant for every
defining word (that takes an argument from the input source).

It is enough to standardize a postfix variant of CREATE and a word that
binds an xt to a name (in a given word list, or in the current word list).


A possible approach to place a new name into a word list:

mould-name ( addr u -- nt )
\ Create a name token for the name identified by the string (c-addr u)
\ and return the created token nt.

fix-name-xt ( xt nt -- )
\ Set the execution token xt for the name token nt

enroll-name-in ( nt wid -- )
\ Place the name token nt
\ into the word list wid.


\ Don't sure that the above part is enough portable


append-wordlist ( xt c-addr u wid -- )
\ Place the name specified by the string (c-addr u)
\ for the execution token xt
\ into the word list wid.

naming ( c-addr u xt -- )
\ Place the name specified by the string (c-addr u)
\ for the execution token xt
\ into the compilation word list.





1. http://forth.sourceforge.net/word/evaluate-with/index.html


--
Ruvim

[Ruvim]

On 2020-04-17 17:07, Anton Ertl wrote:

> a...@littlepinkcloud.invalid writes:
>> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> a...@littlepinkcloud.invalid writes:

[...]

>>>> Changing code on the fly, in response to events.
>>>
>>> Standard Forth does not have anything more than, e.g., C, in this
>>> area, and I am not sure it should.
>>
>> In Standard Forth you can FORGET stuff, reload, and restart. In what
>> way does that not qualify?
>
> It's easier to use deferred words (which is equivalent to what C
> offers), so why would anybody do that? And indeed, AFAIK nobody has
> ever done that, with the following exceptions:
>
> 1) During development, to load code that was manually changed, not in
> response to some non-developer event.
>
> 2) I have read about it being used to implement overlays. If you have
> enough memory (or demand-paging), you do not do that. So this is not
> "changing code on the fly, in response to events".
>

I used code reloading in production.

It is implemented not via old-fashioned FORGET, but via dynamic
dictionaries, when a new dictionary (as a union of the header space,
code space and data space) can be allocated and then freed.

It is used in http server (web application server), for warm reloading —
when the old requests (that are in progress) are still handled by the
old code, and the new requests are handled by the updated code. When the
old requests are completed, the dictionary with the previous version is
freed.


--
Ruvim

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>In Standard Forth you can FORGET stuff, reload, and restart. In what
>>way does that not qualify?
>
> It's easier to use deferred words (which is equivalent to what C
> offers), so why would anybody do that?

How re deferred words equivalent to being able to generate *different
code* from user inputs? You do know about custom extension languages
in Forth, surely, even if you've never written anything like that for
real-world use.

Andrew.

[jimno...@gmail.com]

On Thursday, April 16, 2020 at 2:18:00 AM UTC-4, Cecil - k5nwa wrote:

> Yes I would like to get a cross-compiler that generates x64 code and
> runs in Linux or can run in Wine. I want to do some experiments with
> code optimization so I need to easily be able to easily change the
> generated code as I carry out experiments.
>
> I don't know of any, the several Forth I'm aware of like Vfx and
> SwiftForth carry the whole compiler along with the application, I rather
> not do that as I want to analyze the generated code and if 90% is
> something else not having to do with the application it makes things
> harder.
>

If you don't mind using assembly and have access to a Mac, you can use Diaperglu to do it. Diaperglu has a forth based assembler in it and you can generate files with just the assembly code in it. The problem you'll have is when you go to use the file in Linux and try to call the functions from some other language, like C, you'll have to load the file to a buffer and then manually build the function pointers with the offsets to the functions in the raw code file by hand. Either that or write a tool to build .o files for Linux.

[none albert]

In article <35eb1a25-d380-4bdc...@googlegroups.com>,

Please restrict your lines to 72 char's.
I have two remarks here.
Building .o files is the wrong way around. The python people
do it right. They force c to make a binary file that is
adapted to Python, that can be loaded as is.
So the better way is to add proper Forth headers (for your
Forth that is) to those functions in c and just read that in
as a block with a single patch to link it into the library.
The .o files with their ELF headers... I just have to say
Second Korinthians chapter XIX verse 3 to 5.

The second remark is to the original poster Cecil.
I'm currently directly manipulating assembler code in my
optimiser. It is an illusion that assembler code
affords some level of abstraction above the machine code level
that could somehow be manipulated.
(Or maybe it is. I do something like that in making ciforth
portable, but the mere thought of optimisation goes through
the drain.)

The only reason I'm pretty successful with optimisation
is that for my tool machine code and assembler code are exactly
equivalent.
My assembler ciasdis allows to manipulate machine code
that is exactly specified thanks to my adaptation to the
mnemonics.

In fact I'm manipulating machine code, like in
"
set bit 1 of the first byte of such and such instruction to
use AX's ghost register (extra 8 64 bits register) instead
of AX itself.
"

I use this tool of mine:
https://github.com/albertvanderhorst/ciasdis/releases
ciasdis can test and debug a file with just assembler code.
What is nice, is that the lina64 base system (kernel + library)
contains a 15 screen simplified assembler, that allows
to assemble and run the code.
Although there is some 64 bits stuff there, I still have
to release the latest which allow to disassemble my lina64
executable and reassemble to the exact same binary.

Groetjes Albert

[jimno...@gmail.com]

> Please restrict your lines to 72 char's.
> I have two remarks here.
> Building .o files is the wrong way around. The python people
> do it right. They force c to make a binary file that is
> adapted to Python, that can be loaded as is.
> So the better way is to add proper Forth headers (for your
> Forth that is) to those functions in c and just read that in
> as a block with a single patch to link it into the library.
> The .o files with their ELF headers... I just have to say
> Second Korinthians chapter XIX verse 3 to 5.
>

I couldn't figure out how to generate direct exe files.
Mac Os keeps changing what link commands they
support and don't support in each version. The
latest version uses uleb128 and tries for their
export symbol lists. .o files are easy as they don't
use the new features for the import/export symbol
lists. And using ld to convert the .o file to a
shared library or exe is also easy.

> The second remark is to the original poster Cecil.
> I'm currently directly manipulating assembler code in my
> optimiser. It is an illusion that assembler code
> affords some level of abstraction above the machine code level
> that could somehow be manipulated.
> (Or maybe it is. I do something like that in making ciforth
> portable, but the mere thought of optimisation goes through
> the drain.)

What about branching and pc relative memory access
calculations? Some tools do the math for you. Forth assemblers
sometimes have the Forth style branch instructions in them
which make it a lot easier.

> The only reason I'm pretty successful with optimisation
> is that for my tool machine code and assembler code are exactly
> equivalent.
> My assembler ciasdis allows to manipulate machine code
> that is exactly specified thanks to my adaptation to the
> mnemonics.
>
> In fact I'm manipulating machine code, like in
> "
> set bit 1 of the first byte of such and such instruction to
> use AX's ghost register (extra 8 64 bits register) instead
> of AX itself.
> "
>
> I use this tool of mine:
> https://github.com/albertvanderhorst/ciasdis/releases
> ciasdis can test and debug a file with just assembler code.
> What is nice, is that the lina64 base system (kernel + library)
> contains a 15 screen simplified assembler, that allows
> to assemble and run the code.
> Although there is some 64 bits stuff there, I still have
> to release the latest which allow to disassemble my lina64
> executable and reassemble to the exact same binary.
>
> Groetjes Albert

That sounds pretty cool.

[none albert]

In article <2004a3bb-6311-4fb0...@googlegroups.com>,

<jimno...@gmail.com> wrote:
>
>> Please restrict your lines to 72 char's.
>> I have two remarks here.
>> Building .o files is the wrong way around. The python people
>> do it right. They force c to make a binary file that is
>> adapted to Python, that can be loaded as is.
>> So the better way is to add proper Forth headers (for your
>> Forth that is) to those functions in c and just read that in
>> as a block with a single patch to link it into the library.
>> The .o files with their ELF headers... I just have to say
>> Second Korinthians chapter XIX verse 3 to 5.
>>
>
>I couldn't figure out how to generate direct exe files.
>Mac Os keeps changing what link commands they
>support and don't support in each version. The
>latest version uses uleb128 and tries for their
>export symbol lists. .o files are easy as they don't
>use the new features for the import/export symbol
>lists. And using ld to convert the .o file to a
>shared library or exe is also easy.

As long as you don't need to mess with the actual structure of
.o files, you're good.
You can define forth headers as c-structs and fill
in pointers during linking. The bottom line is that you can
couple from Forth to to a shared library and the only thing to know is
the address of the execution token of a wordlist that
contains "your stuff".

>
>> The second remark is to the original poster Cecil.
>> I'm currently directly manipulating assembler code in my
>> optimiser. It is an illusion that assembler code
>> affords some level of abstraction above the machine code level
>> that could somehow be manipulated.
>> (Or maybe it is. I do something like that in making ciforth
>> portable, but the mere thought of optimisation goes through
>> the drain.)
>
>What about branching and pc relative memory access
>calculations? Some tools do the math for you. Forth assemblers
>sometimes have the Forth style branch instructions in them
>which make it a lot easier.

Easier to program in assembler perhaps, but Cecil nor I
are trying to write an assembler program, but transforming it.
Suppose you transform a piece of code to a smaller piece.
How does having relative addressing at different position in that
code make life easier?
An optimiser splits a program in basic blocks and the first task
is to optimise those, simplifying and getting rid of instructions
normally make thinks smaller. In that kind of circumstance
you have to calculate distances yourself.

[Cecil - k5nwa]

On 4/9/20 10:46 AM, Cecil - k5nwa wrote:
> I want to work on a special  educational project so I'm looking for a
> Forth with special qualities. I will use this software to build other
> Forth Systems with optimizing capabilities.
>
> It needs to be able to generate a stand alone target program that has a
> minimum code in it so it should not include the Forth System in it just
> the desired code. So in essence it's a target compiler for the Pentium
> Family of CPUs.
>
> If that is not available an alternative is that it should be able to
> generate a library file without Forth inside it just the application
> code. Then I can use a C wrapper to execute the software. In some ways
> being able to generate an application only library has a few benefits of
> it's own.
>
> The host system will be a Linux system running specifically Mint 19.3
> x64 Linux. Currently I have been using C in Windows but I would like to
> move away from C and Windows if possible, I'm getting to the point where
> I need to create some complex libraries in C but I rather do it in Forth
> if possible. Extra points will be awarded for a 64-bit solution as Linux
> systems are moving away from 32-bits
>
> I have SwiftForth, and VFX Pro but to use them I have to drag the whole
> Forth development system along and I do not wish to do that if
> avoidable. Both of these include undocumented target compilers so that
> they are hard to work with if not impossible, and they are written so
> basically the entire Kernel needs to be included in the application.
>
> Any ideas on some Forth system that might be usable for this special
> purpose? Otherwise I will have to continue using C or switch to
> something like Lazarus which is more user friendly than C due to built
> in target Graphical IDE components and error checking.
>
> Another remote possible system is Timbre which could be used to generate
> an assembler file that is stand alone but it's an older Windows only
> Forth with little documentation so there will be some difficulties but
> it also has some nice advantages such as it's superb text manipulation
> capability..
>
> Thanks for your suggestions.

I been looking at MinForth V3.4 as a possible candidate to help me out,
so far it's transpiler looks useful. An interesting set of features,
mostly ANS 2012 compatible but with interesting additions such as
Complex Numbers. I'm a HAM Radio Operator and I like the design of
radios which uses a lot of Complex Math in deriving things, a very
useful feature. I also like Software Defined Radios or "SDR" for short
and that involves a lot of Complex Numbers as the radio signals are
represented by a Complex Pair and processed accordingly.

Another candidate will be ciforth but I have not yet looked at it. It
and MinForth share something in common in that they include software to
generate a stand alone Forth which is an area of interest for me.

Reda4 might be of use, I'm not sure yet, I still need to look at it's
compiler that generates target software in Assembler. So far from
reading it is quite different than plain Forth especially in its
conditionals execution and it's methods of looping code, different but
easy enough to learn the new way. I wonder how close to ColorForth this
software is.

The major possible problem with Reda4 is that it is a Windows only
program and I'm moving away from Windows as fast as I can. I'll have to
test it using Wine and see how that works out. Reda4 is useful in
another aspect, I always wanted to try using ColorForth but its limited
display and weird keyboard kept me away, Reda4 takes care of those
issues by not using color but use prefixes instead. ColorForth is
Charles Moore's new direction in simplifying Forth so I would like to
try it out, I liked his previous non-ANS Forth cmForth and used it for a
long time.

So thanks to everyone for their suggestions, your assistance is welcomed.

--
Cecil - k5nwa

[Pablo Hugo Reda]

Cecil

I tell you some things and you can write to me by email if you want to go deeper into others.

Currently :r4 (reda4) has 2 more loyal users, one is me but the other uses exclusively linux, through wine, so it tells me it works well enough.

However this year I started the development of the successor called r3, the main improvement is the further simplification of the loops, especially in the way of compiling this code, and the change to 64bits.

Another change is that the virtual machine is better documented and I use the SDL library, I would allow to compile in Linux with not many changes (in fact I use GCC to generate this virtual machine), it is not the path that I want to follow, but if in Sometime someone wants to make them I will be help to this happen. My path is make the compiler to target linux without the native interpreter.

I am pandemic situation it is giving me a lot of time to advance this project and currently it already has a basic compiler and I am approaching an advanced one. The current compiler draw mandelbrot fractal in 10x the interpreter mode.

:r4 currently has a debugger and profiler, r3 still doesn't, but I think I will have some sort of immediate way to test words and see the stack status in real time shortly.

English is not my native language but maybe doing some kind of chat for jitsi or zoom, it is fun, to see the people who program forth.

best regards,
Pablo

[Chris Curl]

FWIW (not much probably), my little non-ANS, forth-like thing is extremely simple and can easily be made to be a stand-alone thing. I would think it's be pretty good as an educational tool. That said, you probably won't be interested in using it, and that would be totally understandable. But since you said you wanted something that could be stand-alone, and could be built with a limited set of words, it seemed to me that my little project fits those to requirements pretty well. The other positive I have for it is that it uses GCC, and is 100% freeware. It's here:

https://github.com/CCurl/Forth-GCC

Feel free to fork it or even contribute to it if you'd like. And I'd be happy to enhance it or work with or anyone for it to be usable for other people's purposes.

- Chris

[Chris Curl]

Sorry, additionally, I am currently working on a dis-assembler that generates native code for other processors. Anyway ... there you have it.

[Cecil - k5nwa]

I currently have a Forth using C for the VM that I been working on for
the purpose of being portable and just for fun, it uses the Windows OS
but all the OS code is separated and uses Posix to make it easier to
move. It is a token threaded compiler so the code is tiny, 16KB at
present with a 200+ word kernel. The C VM pulls in a Token Threaded File
to expand the VM into a working Forth System. But I would like to go
further, to generate 64-bit native code in a way that is easily
changeable since I would like to work with code optimizing and I'm
interested in experimenting with a 64-bit Linux Version as I'm dumping
Windows.

I have been having serious health issues so work has been slow but
recently I discovered one issue that is fixable that was affecting me
greatly, my CPAP machine had some issues and I was re-breathing my own
exhaust breath causing Carbon Dioxide buildup with resulting headaches,
dizziness, muscle cramps, and could have led to death. That has been
fixed temporarily so I'm beginning to feel much better so I'm eager to
continue with my project.

My next part of this adventure is to be able to generate 64-bit code for
Linux, ideally it could serve as a Target Compiler to generate stand
alone code, stand alone meaning no Forth compiler would be needed in the
target application. There are many ways of approaching this problem so
I'm willing to try various approaches. At present I am looking at
MinForth with it's transpiler converting Forth to C, a workable solution
that could be expanded on to a Forth to Assembler version.

Another project I would like to work with is to examine ColorForth via
Reda4 it also has a target compiler built in to convert Forth Words to
Assembler code so that relates to my first project. It's main problem is
that it is a Windows only project but it can run in Wine which makes it
possible to work with it in Linux.

I will look into your Forth later on, right now I have three Forth
systems to look at in the following order;

1. MinForth, I'm about to look seriously at it's transpiler and see how
easily changeable it is. I also looked useful to migrate Forth to other
systems since the generated program is in C and the C compiler is
available for just about every CPU in existence.

2. Reda4, interesting because I want to look at something similar to
ColorForth but also interesting in that it comes with a Forth to
Assembler target compiler. So again how easily changeable is it?

3. ciforth, described as a Forth system generator which sounds promising
for my purposes, I have not looked at it all so I am not aware if it has
the possibility to do what I want but it sound very interesting, 32-bit
and 64-bit capability, and able to generate code for Windows and Linux,
with optimizing ability, mainly it will be a question of how easy will
it be to change the code and optimization,

Hopefully now that I feel better than before I can put some effort into
this project whose purpose is to learn, and have some fun.


--
Cecil - k5nwa

[Chris Curl]

When your first post mentioned "educational", my mind went to teaching young people the concepts of a computer system, as opposed a hobby project for a seasoned adult familiar with forth and computer systems. So, yeah, sorry for going off-topic a little.

You may still find it to be an easily changeable change that you could build on, if your preference it to start more from the beginning.

But you will most likely find my little thing to be much too immature and lacking in the features you are looking for.

[Cecil - k5nwa]

It's a project for self learning, of course once it works to my liking
then I will use it from then on, at that point then it will become a
more practical project.

I will look at your project also, another possibility is once I've
looked at all the software I will then write my own from scratch, but
I'm hoping that one of these system will be usable and eliminate writing
it all from scratch. I have in the past used Assembler, Basic, C,
Pascal, and a macro processor to create Forth systems and although I
feel very comfortable with C and Pascal the one I had the most fun with
was using a Macro Processor to create a Forth Target Compiler, I've used
Stage2 and Gema in the past and they were fun to use.

I have for many years found writing compilers and optimizers a task that
I like, I have in the past written several C and Pascal compilers the
last one a Full K&R C compiler with optimizations and support for many
CPUs. I gave up on C a long time ago but lately I've returned to using
it for my little educational projects. The C and Pascal compilers I
wrote are pretty conventional using Recursive Decent and generating code
that was then optimized. I have written several Forth compilers but they
were off the beaten path, none were written in Forth itself, using
Assembler, C, Basic, and Macro Processing languages, so I don't mind
breaking established norms, the current project is the most "normal"
Forth System written in C as many other Forth Systems with the bulk in
Forth cross-compiled to a token threaded file included in by the C
compiler so it's 5% C and 95% Forth.

Anything is possible for the "final" solution including re-writing the
whole thing in C, Pascal, or Forth or a mixture of these. I am retired
due to health issue so I have these projects to keep me busy and active

[A. K.]

BTW and FWIW I just added INLINE as new transpiler command. Drop me a pm at
minf...@gmx.net and I can send you a copy.

Before rewriting the transpiler (and mf3.h) to spill out asm code instead of C
it might be worth while to prototype with C built-in assembler.

By design MinForth execution tokens are cells containing real addresses i.e.
function pointers. They could be converted without much ado to direct
assembler calls.

I am hoping that you'll get enough fun from your projects to keep you in good
mental and better physical shape. I know from my own experience how important
this is....

Andreas

[Cecil - k5nwa]

> By design MinForth execution tokens are cells containing real addresses i.e..

> function pointers. They could be converted without much ado to direct
> assembler calls.
>
> I am hoping that you'll get enough fun from your projects to keep you in good
> mental and better physical shape. I know from my own experience how important
> this is....
>
> Andreas
>

For now I will not make any changes but if things workout, I will keep
the original transpiler, but duplicate it and modify a second copy to
translate to x64 assembler.

A lot of things have to workout before I decide to make any changes. I
want to do that so I can test code with the C transpiler and figure it
all out before generating assembler code, the last thing will be to work
on optimizing the assembler code.

In the documentation I see nothing about vocabularies, are they
supported or did I miss something?

--
Cecil - k5nwa

[A. K.]

You are welcome.

Missed ;-) it's in the "doc" file § 2.7.8

Try it in MF (with search.mfc in the build)

VOCABULARY MYVOC
VOCS \ shows all wordlists
ALSO MYVOC DEFINITIONS
ORDER
\ et cetera

[Cecil - k5nwa]

Vocabularies can be very useful, I specifically was interested for use
with creating a code stripper but since one is available in the
transpiler then there is no harm either way.

So far MinForth will be very useful for my project it has;

Code stripper - very useful avoids confusion on whose code is running.
Full Source Code - it would be useless without it.
Generates C code - very useful, if easily changed.
32-bit/64-bit - very useful Linux is moving to 64-bits only.
Windows/Linux - nice to have, Windows will soon be gone from my use.
Complex numbers - I will have use for that for radio applications.
ANS2012 compatible - neutral on that, mostly anything reasonable would do.

--
Cecil - k5nwa