32-bit/64-bit Pygmy Forth for Linux, MacOS, Windows

[Frank Sergeant]

32-bit/64-bit Pygmy Forth for Linux, MacOS, Windows

For your Forthing pleasure, I have recently posted my new Pygmy Forth
at http://pygmy.utoh.org/pygmy64.html.

It is in the spirit of Pygmy Forth for DOS, I feel, but quite
different.

The new version is implemented in Python and, thus, runs everywhere
that Python runs.

Python is its assembly language and it is easy to write CODE words.
Here is a simple example with no stack effects:

CODE HELLO
print ("Hello")
END-CODE

Source code can be loaded from either a text file or from a pseudo
block file.

For Emacs fans, it comes with pygmy-mode to make working with pseudo
block files especially convenient. PgDn, PgUp (or C-v, M-v) move from
block to block, narrowing the display to show just the current block.
S-TAB cycles through the 3 visibility settings (as in org-mode). For
example, show just the first lines of each block or the entire file.

The manual can be read in a web browser (HTML), downloaded as a PDF,
or downloaded as an eBook (either Mobi for Amazon Kindle, or EPUB for
other eReaders).

MIT license.

As always, I look forward to comments, corrections, suggestions.


--
Frank
http://pygmy.utoh.org

[Cecil Bayona]

Thanks for the release, Pygmy has been one of my favorite versions of
Forth but working with it has become somewhat of a challenge so I have
to run it in VirtualBox Win7 i386 environment since Windows 10 will not
run DOS applications anymore.

I been wanting a modern version of it in Assembler but I hate having to
learn assembler for the Intel family so I never done anything about it,
it looks like you also do not want to bother either as this version uses
Python. I'm not sure how fast will this Pygmy will be as it is using
Python as the core language I would guess to be way way slower than
assembler.

Lately I started on a learning project of creating an eForth in the Go
language as the core language which at least has an optimizing compiler,
but I have stopped working on it due to health issues. I had so much
fluid in my lungs that the doctors were surprised that I was still
walking around, it's slowly getting better but they still do not know
the cause, but at least the two major issues lungs and heart are fine so
it's something else. So after a month I might be well enough to continue
my projects and I might consider a Python to Go translation so it will
be magnitudes faster.

Thanks again I will try it soon and look into the possibility of
translating it to Go or C as my health improves.

--
Cecil - k5nwa

[Helmar Wodtke]

Nice thing! I like that you target something else than assembler. To bring Forth back to use today, I think it's needed to target systems that are not "bare metal". As you use Python you have a complete "JIT" that is faster than one might think first. I currently experiment with awk as "assembler": https://bitbucket.org/helmwo/txfwia/src - awk is not "dynamic", so I dont have this JIT-feature for now. This makes it slow. It's more a toy, as one goal is to have it running on original-awk (as slow as this might be). It works also as C-version, that is way faster and can do some things easier (as to open bi-directional pipes). I have plans to implement as Javascript-version if the other implementations are more mature. With Javascript I do also have the possibility for a JIT-compiler. The JIT will work on base of the threaded code in background (I made something like this before for C/Assembler).

> Thanks for the release, Pygmy has been one of my favorite versions of
> Forth but working with it has become somewhat of a challenge so I have
> to run it in VirtualBox Win7 i386 environment since Windows 10 will not
> run DOS applications anymore.
>
> I been wanting a modern version of it in Assembler but I hate having to
> learn assembler for the Intel family so I never done anything about it,
> it looks like you also do not want to bother either as this version uses
> Python. I'm not sure how fast will this Pygmy will be as it is using
> Python as the core language I would guess to be way way slower than
> assembler.
>
> Lately I started on a learning project of creating an eForth in the Go
> language as the core language which at least has an optimizing compiler,
> but I have stopped working on it due to health issues. I had so much
> fluid in my lungs that the doctors were surprised that I was still
> walking around, it's slowly getting better but they still do not know
> the cause, but at least the two major issues lungs and heart are fine so
> it's something else. So after a month I might be well enough to continue
> my projects and I might consider a Python to Go translation so it will
> be magnitudes faster.
>
> Thanks again I will try it soon and look into the possibility of
> translating it to Go or C as my health improves.

C is not dynamic. So the current cool implementation technique to use the dynamic language as assembler/JIT is not possible. Go will have similar problems. For C you can try Tiny C Compiler (tcc) from Fabrice Bellard - it has a library to dynamically compile C-code that is simple to use and more lightweight than say LLVM or similar.

Regards,
-Helmar


> --
> Cecil - k5nwa

[Frank Sergeant]

Cecil Bayona <cba...@cbayona.com> writes:

> On 11/9/2017 2:26 PM, Frank Sergeant wrote:
>> 32-bit/64-bit Pygmy Forth for Linux, MacOS, Windows
>>
>> For your Forthing pleasure, I have recently posted my new Pygmy Forth
>> at http://pygmy.utoh.org/pygmy64.html.

> Thanks for the release, Pygmy has been one of my favorite versions of
> Forth but working with it has become somewhat of a challenge so I have
> to run it in VirtualBox Win7 i386 environment since Windows 10 will
> not run DOS applications anymore.

Yes, I fear it has become more and more outdated. I sometimes run it in
a DOS box under Linux using dosemu.

> I been wanting a modern version of it in Assembler but I hate having
> to learn assembler for the Intel family so I never done anything about
> it, it looks like you also do not want to bother either as this
> version uses Python.

Well, it is more than that for me. Many years ago, I did port the
16-bit Pygmy for DOS to 32-bits, for a client, if I recall. I never
developed it further.

It's more that I don't want to be tied to a particular CPU or operating
system, and yet still have something similar (in some ways) to the older
Pygmy.

> I'm not sure how fast will this Pygmy will be as it is using Python as
> the core language I would guess to be way way slower than assembler.

I imagine it is slower, but whether that would be worth worrying about
would depend on the specific application. I wouldn't want to worry
about that prematurely.

If it really turned out to be a problem in a specific application, we
might consider these steps:

- the bottlenecks could be re-written in C (CFFI
https://cffi.readthedocs.io/en/latest/)

- the bottlenecks could be re-written in C directly in a CODE word
using Weave (http://scipy.github.io/old-wiki/pages/Weave)

- use the SciPy libraries (https://scipy.org/)

- the program could be run in PyPy (http://pypy.org/) rather than in
cPython

- the bottlenecks could be offloaded to a server process (written in
anything) and the main Forth program (i.e., running Python) could
communicate with the server process

- etc.

> Lately I started on a learning project of creating an eForth in the Go
> language as the core language which at least has an optimizing
> compiler, but I have stopped working on it due to health issues. I had
> so much fluid in my lungs that the doctors were surprised that I was
> still walking around,

I'm sorry to hear of your health problem. Here's to your speedy and
full recovery.

> it's slowly getting better but they still do not know the cause, but
> at least the two major issues lungs and heart are fine so it's
> something else.

I would have thought fluid in the lungs was a strong indication that the
lungs were *not* fine. ;)

> Thanks again I will try it soon and look into the possibility of
> translating it to Go or C as my health improves.

Great. I'd like to hear how that turns out.


--
Frank
http://pygmy.utoh.org

[JUERGEN]

Frank, is there an indication if this Forth will run as well under MicroPython? As you know I am into having the microbit hardware and soon the Calliope board as a basis. Until now, Mecrisp Forth is running on both, but from the beginning I wanted this bridge via Python - which seems to be what you have created.

[Frank Sergeant]

JUERGEN <epld...@aol.com> writes:

> On Thursday, November 9, 2017 at 8:26:49 PM UTC, Frank Sergeant wrote:
>> 32-bit/64-bit Pygmy Forth for Linux, MacOS, Windows

> Frank, is there an indication if this Forth will run as well under
> MicroPython?

I think there is a good chance that it will. I hope you will try it and
let me know for sure. The version of Python I have been using is 3.6.2
and MicroPython may be based on a slightly earlier version (maybe 3.4),
but I doubt that will make any difference.


--
Frank
http://pygmy.utoh.org

[Paul Rubin]

JUERGEN <epld...@aol.com> writes:
> Frank, is there an indication if this Forth will run as well under
> MicroPython? As you know I am into having the microbit hardware

MicroPython is a complete implementation of Python and it's likely that
it can run any reasonable Python program with some possible porting
effort. However, I'd still consider the Microbit doubtful as a target
without trying it first: the Microbit uses a mangled and squished
version of MicroPython due to its limited ram, and its ability to run
complicated Python programs is constrained.

I'm not saying it won't work, but just that to find out, you really have
to try it. I like to hope they'll do a revised Microbit with more ram
sometime, and this issue will go away.

[Paul Rubin]

Frank Sergeant <fr...@pygmy.utoh.org> writes:
> For your Forthing pleasure, I have recently posted my new Pygmy Forth
> at http://pygmy.utoh.org/pygmy64.html.

Frank, this is really cool, and I remember admiring the earlier Pygmy
Forths though I didn't try to use them. Also, by coincidence, I'm
fooling around with something inspired by your old 3-instruction Forth.

Regarding pygmy64, I wonder:

1) What do you use it for, where you prefer it over using Python directly?

2) Do you think of adding Python code packaging for "pip install" etc.?
Or if someone else did that, would you want to use it?

3) Do you plan to have a code repo or just stay with a zip file?
I'd be happy to host a git repo for you if you want that.

4) Is it ok with you if I incorporate pygmy64 into the host side of the
thing I'm currently coding? Would you want any resulting patches?
Right now my thing's status is just an amusement project that I could
abandon at any moment, but there's a chance I can get it doing
something useful first.

[Lars Brinkhoff]

With Frank Sergant's blessing, I have mirrored his Forths to GitHub:

https://github.com/utoh

[Julian Fondren]

On Thursday, November 9, 2017 at 2:26:49 PM UTC-6, Frank Sergeant wrote:
> 32-bit/64-bit Pygmy Forth for Linux, MacOS, Windows
>
> For your Forthing pleasure, I have recently posted my new Pygmy Forth
> at http://pygmy.utoh.org/pygmy64.html.
>
> It is in the spirit of Pygmy Forth for DOS, I feel, but quite
> different.
>

https://github.com/search?utf8=✓&q=forth+in+python&type=

Here you go, a whole bag of Forths-in-python that implement a few dozen
core words while also adding incompatible 'innovations' aimed at
less Forth-in-python than pythonlike-Forth-in-python.

Other scripting language implementations are also available.

You have a stylish webpage and the old Pymgy was a rich and useful package.

This however is trash and the praise for it exposes clf as a mess of LARPers.

[Cecil Bayona]

On 11/9/2017 2:26 PM, Frank Sergeant wrote:

This morning I was looking at your documentation, and will actually try
and play with it later in the week after I upgrade to Python V 3.6.

For Pygmy I used to have a library of code routines coded and tested for
it but I have not been able to find it. I apparently had it on PCs that
were replaced several times and sometime I wiped out the backups not
realizing that not all the software was too old. The most desired item
was a set of words to implement a B+ Tree database routines they were
quite useful and fast. There was other words for creating and displaying
menus, and date/time and string processing.

--
Cecil - k5nwa

[Lars Brinkhoff]

Frank Sergeant wrote:
> For Emacs fans, it comes with pygmy-mode to make working with pseudo
> block files especially convenient. PgDn, PgUp (or C-v, M-v) move from
> block to block, narrowing the display to show just the current block.

FYI, two other Emacs commands that work with pages are C-x [ and C-x ],
backward-page and forward-page. They normally the ASCII form feed
character as a page separator.

[Paul Rubin]

Julian Fondren <julian....@gmail.com> writes:
> This however is trash and the praise for it exposes clf as a mess of
> LARPers.

Oh come on, it's hobby software and people writing code for their own
enjoyment can write whatever they want. I'm not sure why Frank would
want to write desktop apps in it, but I have an interest in using it as
the host side of a tethered Forth that I'm fooling around with. That
lets it have a traditional Forthy UI while making the implementation
very easy.

[Frank Sergeant]

Paul Rubin <no.e...@nospam.invalid> writes:

> Frank, this is really cool, and I remember admiring the earlier Pygmy
> Forths though I didn't try to use them. Also, by coincidence, I'm
> fooling around with something inspired by your old 3-instruction Forth.

I noticed your thread about that. I don't know whether it would help,
but Riscy Pygness (https://github.com/utoh/riscy-pygness,
http://pygmy.utoh.org/riscy/) uses Tcl on the host side so it might be
possible for you to adapt that or to get some ideas from it.

> Regarding pygmy64, I wonder:

> 1) What do you use it for, where you prefer it over using Python
> directly?

It's brand new. I haven't used it yet for anything except briefly for
testing. I wanted it because I enjoyed using Pygmy for DOS in the "old
days", but for a long time it hasn't seemed to suit the modern computer
environments. I thought perhaps Pygmy64 (I guess that's its name now
even though it also runs in 32 bits) would give me some of what I have
missed. Will I start using it? I don't know.

> 2) Do you think of adding Python code packaging for "pip install" etc.?
> Or if someone else did that, would you want to use it?

I like the idea of it but I haven't looked into how to do it. Yes, if
someone else would do it and perhaps explain to me how, I would be happy
to have it.

> 3) Do you plan to have a code repo or just stay with a zip file?
> I'd be happy to host a git repo for you if you want that.

Thanks to Lars, the code repo is already set up: See the utoh group at
https://github.com/utoh. It has all 3 projects.

> 4) Is it ok with you if I incorporate pygmy64 into the host side of the
> thing I'm currently coding? Would you want any resulting patches?
> Right now my thing's status is just an amusement project that I could
> abandon at any moment, but there's a chance I can get it doing
> something useful first.

Yes and yes.


--
Frank

[Cecil Bayona]

On 11/9/2017 2:26 PM, Frank Sergeant wrote:

I have been a fan of Pygmy for many years and lamented when it became
problematic that it basically supported DOS and that became obsolete. I
also must admit that I'm not a fan of implementing Forth with a high
level language much less an interpreted language like Python. Thinking
back on it those preferences are not based on much other than personal
preferences rather than based on practicality. So allow me to change my
perspective since I value Pygmy far more that some lame excuse, so I had
a couple of questions for you.

1. Is this a re-write from scratch so it might end up different than
Pygmy V 1.7?

I hope not, we have enough ANS compatible Forth so another one is not
needed, also being ANS compatible means adopting all the warts and
limitations that it has a big minus for me, it also means giving up of
some of the unique features of Pygmy.

2. Do you have available the source to Pygmy V1.7 as a plaintext file?
If not I would need to write some sort of converter because I'm
interested in the original code as a text file so I can look and work on it.

I am interested in Pygmy for Windows for one purpose to write Forth
cross-compilers for embedded CPUs.

Thanks
--
Cecil - k5nwa

[hughag...@gmail.com]

On Saturday, December 23, 2017 at 8:54:25 PM UTC-7, Cecil - k5nwa wrote:
> I am interested in Pygmy for Windows for one purpose to write Forth
> cross-compilers for embedded CPUs.

Primarily, ANS-Forth fails to support cross-compilers because LITERAL is not vectored --- this is necessary so TARG words can have literal numbers.
You need DLITERAL and FLITERAL to be vectored too, if doubles and floats are going to be supported in TARG code.

A work-around is to write your own outer-interpreter rather than use the built-in outer-interpreter (that uses the built-in LITERAL).
This is problematic in ANS-Forth because FIND is ambiguous. For a long time I thought this prevented ANS-Forth from supporting a custom outer-interpreter.
The disambiguifiers fix this problem though.

If Frank Sargent makes LITERAL vectored, he will support cross-compilers in an easy and obvious way. Writing a custom outer-interpreter is over-complicated.

Also, the following words are very useful:
LATEST ( -- xt ) \ the xt of the latest definition
VOCNO@ ( xt -- vocno )
VOCNO! ( vocno xt -- )
IMMEDIATE@ ( xt -- flag )
IMMEDIATE! ( flag xt -- )
SMUDGE@ ( xt -- flag ) \ the level-0 smudge flag (set if word is smudged)
SMUDGE! ( flag xt -- ) \ the level-0 smudge flag
PRIVATE ( -- ) \ set the level-1 smudge flag
PRIVATIZE ( -- ) \traverse the dictionary and find words with level-1 smudge flag set; clear it and set the level-0 smudge flag
LOCAL ( -- ) \ set the level-2 smudge flag
END-MODULE ( -- ) \ traverse the dictionary and find words with level-2 smudge flag set; clear it and set the level-0 smudge flag
All of these words should be trivial to implement, but they do require carnal knowledge.

Also it is useful to support rquotations. This should be trivial to implement, but it does require carnal knowledge.
I have rquotations working in VFX and SwiftForth. VFX was trivial --- SwiftForth was more complicated due to SwiftForth's bizarre internal workings.
In general though, this should be trivial to implement.

I'm not enthusiastic about a Forth written in Python because it is going to be slow.
A cross-compiler doesn't need to be fast though, so if cross-compilation is the "killer application" then Frank's Forth should be okay.
The advantage of using Python is that there are a lot of code-libraries available ---
it should be possible to make a GUI IDE fairly easily --- I can't think of any other code-library that would be useful for a cross-compiler.

ANS-Forth was a failure because it didn't support cross-compilation. This was most likely done on purpose.
Forth Inc. and MPE wanted to prevent cross-compilers from being written in ANS-Forth so only they would be able to write them using carnal knowledge.
Now Emperor Pelc is the chair-person of Forth-200x and he continues to strive to prevent cross-compilers from being written in Forth-200x.
Micro-controllers are where the money is --- this is why the Forth-200x committee wants to prevent micro-controller code development using Forth-200x.

I think it would be great if Frank provided support for cross-compilation in his Forth system. I would use it. :-)