Now, why did I burn two days on implementing AES?

[The Beez]

Hi 4tH-ers!

Well, because it has been three years since the latest release - and I want to get another release out. And second, I had decided there would not be a new release without AES. Why? Because I had created a ticket for it.

I don't know if I mentioned it before, but this is the third AES Forth implementation with 128-bit encryption - and the second with 192-bit and 256-bit encryption. So I think it's cool. I haven't tried yet to port it to Forth, but if I did my work properly, it should not be too difficult.

What surprised me is that I never addressed the subject of encryption - although we did have two TEA variants and RC4 available. So, today I added that section to the manual. But gee - I really had to dig in my memory to get all the data required out! I haven't touched this stuff in ages. Especially RC4 was a challenge.

Note other subjects have been touched as well, like .BMP support, .SVG support, 30 bit fractions and locals. Even investigating and updating all lists took some time - and still the manual isn't done yet, although we're coming there.

I think it will be a summer release - but we'll see. Anyway, now you know it's coming. ;-)

Hans Bezemer

[p da]

great news!!    a new release is always something to celebrate 

Thanks

--
You received this message because you are subscribed to the Google Groups "4tH-compiler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 4th-compiler...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/4th-compiler/af385bf6-0327-4725-b7f4-2fb9e222992fn%40googlegroups.com.

[The Beez]

> great news!!    a new release is always something to celebrate 

Sure it is - but we're not there just yet ;-)

Still, it's shocking to see what things are now not available to the average user:

Words
• The words BINARY, SPIN and CLIP were added.

Functionality
• The words BINARY, SPIN and CLIP were added.
• A library to write Media Wiki tables was added.
• A library to write JSON files was added.
• A library to format ISO 8601 dates was added.
• A library to encrypt data according to the Advanced Encryption Standard was added.
• A co-routine implementation using a round-robin buffer was added.
• Several integer square root libraries were added.
• Extensive SVG support was added.
• Loading and saving .BMP bitmaps was added.
• A simple printf() like formatter was added.
• A double word binary shift was added.
• Left and right binary rotate were added.
• A 30-bit wide fixed point calculation library was added.
• INSTR() compatible string searching libraries were added.
• New randomizers were added.
• A floating point word to calculate the perimeter of an ellipse has been added.
• A new local variable library has been added.
• New preprocessor libraries were added to support local variables - up to full Forth 200x compatibility.
• A vector class has been added to FOOS.

Bugfixes
• A bug in exec_4th.c failed to initialize the picture buffer correctly.

I suppose you can get by without, but I think I'd be quite annoyed when I wouldn't be able to use those. The list is even worse for uBasic/4tH users..

Hans Bezemer

[p da]

Impresive list of features!

I'm very interested in your implementation of coroutines, may you explain the internals of this?:

• A co-routine implementation using a round-robin buffer was added.

Sure It would be a great video for back and forth series 

Regards

To view this discussion visit https://groups.google.com/d/msgid/4th-compiler/1373d9c6-6771-4452-899c-e3eb23542c41n%40googlegroups.com.

[Guy Mengel]

Wow Beez,

Thanks.. just getting started here with your documentation and playing with the compiler.. ex long time programmer/engineer..

what fun!  thanks for all your HARD work.. PS.. I work a bit with embedded 4th on microcontrollers (Flashforth).  Scamp3E/3.

I like your way of thinking and going through step by step why things are so..   I will download and install new version when ready.

Best regards

Guy in rural VA. (N1GMM)

[The Beez]

Didn't I already do one on co-routines? ;-)

https://youtu.be/FH4tWf9vPrA

Hans Bezemer

[The Beez]

Hi Guy!

Welcome to the club! And we're not working too hard down here, this thing is a project of more than thirty years - so we took our time ;-)

I guess you already figured out this is quite a different beast than "ordinary" Forth compilers - and maybe you may find a few surprises.

No worry, if you have any questions, don't hesitate to ask. I think we're usually pretty quick on the ball here.

Note an upgrade is usually pretty painless. If there are any issues you'll find them in the first few pages of the manual, like:

• If you defined BINARY within your program (typically : BINARY 2 BASE ! ;) it will fail to compile. Simply remove the definition. If such definition is part of a wordlist, replace it by : BINARY_ BINARY ; and change the entry in the wordlist accordingly;
• The library ffl-frc.4th has been renamed to frc.4th. If you used this library in any of your programs, change the offending INCLUDE accordingly.

So don't let that stop you. Sure, we've had our share of design errors, but I hope those are behind us now.

Anyways, I hope your experience with 4tH will be a good one..!

Hans Bezemer

[Timur]

Hans, I remembered that I forgot to apologize to you. I once (may be 2 years ago) wrote a message saying that "C++ is much more powerful than your library". It was my stupidity that I realized only now. I am very sorry!  Timur

понедельник, 31 марта 2025 г. в 17:54:08 UTC+3, The Beez:

[The Beez]

Oh dear Timur, you don't have to! :)

By some measure, you might even be right. It hasn't registered with me - but it is appreciated nonetheless.

I consider you to be a welcome member to the community, but please - don't lose any sleep over this! You didn't hurt my feelings! ;-)

Hans Bezemer

[p da]

El mié., 2 abr. 2025 15:30, The Beez <the.bee...@gmail.com> escribió:

Didn't I already do one on co-routines? ;-)

https://youtu.be/FH4tWf9vPrA

I remember to watch that video and also I had to think carefully all stacj movements to understand the words involved but I'd like to see the technique using round Robin buffer, the video was mainly about mangling with the return stack with smart defined words 

Also I remember to wonder myself if that smart use of return stack would have problema with other words playng with the return stack

Could you discuss all this in a more detailed manner? Is there any "safer" way to achieve coroutines?

Greets

 

To view this discussion visit https://groups.google.com/d/msgid/4th-compiler/80b300b8-b360-4f74-9bd9-274f922a4aban%40googlegroups.com.

[The Beez]

Frankly, I got it to work - but I don't have a clue how it ticks! ;-)

The only thing I'm quite confident about is that it ticks correctly (I dived in deep enough for that).

I really have to dive into it, but again (frankly) at the moment I have other priorities like getting out a new vid and releasing this version.

That doesn't mean "never", but the old-timers know it may take a while..

Hans Bezemer

[The Beez]

> Is there any "safer" way to achieve coroutines?

Safe? Forth? Well, you can't get any safer than 4tH, because that operates Forth in a well guarded sandbox - but don't worry, I know what you mean.

Well, if you know what's currently on the return stack - and respect that - you can still use the return stack.

However, if you think you can temporarily store a value on the return stack - and then use thingies like YIELD without balancing the return stack first, you're definitely asking for trouble.

But the same applies when using DO..LOOP.. Or a method in FOOS. If you're not confident, refrain from using such facilities until you are. 

On the other hand, you will never gain such confidence until you do - and fall flat on your face a few times. 

Tools like RCHECK.4TH and RDEPTH.4TH may help you (I know I used them for co-routines research).

However, contrary to C everything is open to you. You can open up a panel and see how the machine is working.. That's Forth.

It may not seem related, but uBasic/4tH shares some that same characteristic. It has a user-facing stack, which is also used to pass parameters. Strings are pointers - and you can examine them (try that in any other BASIC).

Coming from C, I learned that KNOWING how things work helps you becoming and being a better programmer. I don't say you shouldn't hide things under the hood - but make sure you can open it.

Hans Bezemer

On Thursday, April 3, 2025 at 8:49:07 AM UTC+2 euke...@gmail.com wrote:

[The Beez]

Okay, I got a bit wiser this time.

First, this thing was created to fulfill a Rosetta Code task. I like that site and from time to time, I contribute to it as well.

This is the page: https://rosettacode.org/wiki/Synchronous_concurrency#

So it seems the library (found here: https://theforth.net/package/co/0.0.1) was just a quick hack. Now, the circular buffer is nothing special (compare: BUFFER.4TH). We start off with two pointers, HEAD and TAIL - which are identical at startup.

The moment an element is added, it's added at the HEAD. When an element is removed, it's removed from the TAIL - hence a "First in, first out" (FIFO) stack. Since the return stack is essentially a LIFO (Last in, first out) stack, I'd say that's a significant difference. ;-)

: ADJUST   ( a--a')  START - CQ# 1- AND START + ;
: PUT      ( n-- )   TAIL @ TUCK ! CELL+ ADJUST TAIL ! ;
: TAKE     ( --n )   HEAD @ DUP @ SWAP CELL+ ADJUST HEAD ! ;
: CQINIT   ( --  )   START DUP HEAD ! TAIL ! ;
: NOEMPTY? ( --f )   HEAD @ TAIL @ <> ;

Now, COINIT simply initializes the pointers HEAD and TAIL. Since they are identical, NOEMPTY? renders false.  PUT and TAKE are the equivalent of >R and R>. ADJUST wraps the "stack" so when a pointer reaches the "end", it returns to the beginning.

To illustrate that, let's consider a favorite word of mine (I use it often to annoy Python programmers. I know, I'm not a nice guy ;)

: am12pm 11 + 12 mod 1+ ;

Now lets do 24 hours:

: 24h cr 25 0 do ." ( "  i . i am12pm . ." )" loop cr ; 24h
( 0 12 )( 1 1 )( 2 2 )( 3 3 )( 4 4 )( 5 5 )( 6 6 )( 7 7 )( 8 8 )( 9 9 )( 10 10 )( 11 11 )( 12 12 )( 13 1 )( 14 2 )( 15 3 )( 16 4 )( 17 5 )( 18 6 )( 19 7 )( 20 8 )( 21 9 )( 22 10 )( 23 11 )( 24 12 )

So that's how Americans name their hours. The wrapping here is done by MOD, the wrapping in ADJUST is done by a bit mask (that's why the buffer needs be "a power of two").

Now, those are all the words concerning the FIFO ring buffer.

Now I'm pretty sure that a co-routine is registered by CO:. I further assume that CO transfers control to the next one in line. I think ;CO is the equivalent of GRAB - but this is where it gets murky. I think GO is the thing that controls this "round robin" kind of cooperative multitasking.

What I did learn from the Rosetta Code page is that I've not used the example correctly. It should be invoked like this: cat synco.4th | pp4th -x synco.4th

And indeed, that works as advertised:

\ synco.fs Synchronous concurrency for RosettaCode
\ co.fs Coroutines by continuations.

include lib/anscore.4th
include lib/ansfile.4th
include lib/co.4th

\ * CHANNELS LEXEME

2 ARRAY CHAN

: CHAN?  ( a--f )   DUP @ SWAP CELL+ @ XOR ;

\ * READER LEXEME
 
4096 CONSTANT L#
L# STRING Line

: READER
CO:
BEGIN                          \ Press CTRL-D or CTRL-Z to stop
Line L# STDIN read-line THROW
WHILE                          \ Use the length of the string read!
>ZERO Line CHAN 2!
CO
REPEAT DROP
Line DUP CHAN 2!

\ -- Wait for report back
BEGIN CO CHAN CHAN? UNTIL

\ -- Have it, show and go
CR S" -------" TYPE
CR S" LINES: " TYPE CHAN @ ?
;

\ * WRITER LEXEME

VARIABLE X
: WRITER
CO:
BEGIN
CHAN CHAN?
WHILE
CHAN @ COUNT TYPE CR
1 X +!
CO
REPEAT

\ -- Chance to stop other writers
CO

\ -- First of writers reports back
\ -- the shared global counter
CHAN CHAN? 0=
IF
0 X CHAN 2!
CO
THEN
;

\ * RUNNER

CQINIT 0 X ! READER WRITER ( WRITER WRITER :-) GO CR DEPTH .


-------
LINES: 65

$

So, that's what I got for you. Note this routine is also playing with the return stack - so I'm not sure it is actually "safer"..

Hans Bezemer

[The Beez]

Well, this is how it works. As I suspected, CO: sets up the co-routine, CO makes it switch, GO controls the entire "round robin" contraption.

See the enclosed files for details.

Hans Bezemer

[The Beez]

BTW, I traced "Humptydumpty" - which is the handle of Ouatu Bogdan Ionut.

He's listed here: https://codeberg.org/clausecker/mecrisp-stellaris/src/tag/v2.4.5/CREDITS

Hans Bezemer

[p da]

I was playing around with this example in https://theforth.net/package/co/0.0.1  specially with coroutines implementation and maybe losing my time because I can't fully understand how it works

The code says it''s an implementation of couritines based on continuations, my assumption is it understand a continuation as a return address in the return stack and so the coroutines words are all about storing return address in the circular queue so CO: init coroutine adding it (return stack address) at the end of the queue , ;CO ends the coroutine getting out the queue head  and CO does the coroutine switching (commonly named yield) just adding current return address in return stack to the end of the queue and removing the first return address from head of the queue to store it in return address so at the exit of CO it will return to first coroutine. Finally GO simply continues extracting routines (return addresses) from the queue after exhaust.

So it's a matter of playing with return stack and since what there's in return stack are address to return back you can see them as continuations and thus jump to one word or another.

What I doesn''t fully understand (sure due to my basic knowledge of forth internals) is how to start all these dancing,  in the example the starting point is the line:

\ 0 X ! READER WRITER ( WRITER WRITER :-) GO CR BYE

since I asume ( ... ) is a comment you can rewrite it as

\ 0 X ! READER WRITER GO CR BYE

since reader contains a CO:  and a CO in a loop I understand that when reader is first called it adds the current return address to the queue (by CO:) but what 's exactly that return address?  I suppose is the address of the word calling reader, that is the main loop because is invoked from terminal, so it should be "interpret" or maybe "quit", and inside the begin repeat loop there's a CO but my understanding is there's no variation in return stack since we're in NEXT loop without touching return stack (even when initiating a new next level for a colon word since return stack is adjusted properly)  So, between what is switching CO inside the loop in reader?  (it's supposed to switch to writer but didn''t see how)

When executing writer we have a similar situation but at least we have something in the queue

I know this is not stricty related to 4th so I will understand if you prefer not to reply to this message.

regards

To view this discussion visit https://groups.google.com/d/msgid/4th-compiler/f51dfe0e-6578-4040-b61c-32fb3e8ea6d1n%40googlegroups.com.

[The Beez]

Hi,

Well, I made a vastly simplified example to crack it:

include co.4th

: .-   CO: BEGIN 8 emit [char] - emit sync CO REPEAT ;
: .\   CO: BEGIN 8 emit [char] \ emit sync CO REPEAT ;
: .bar CO: BEGIN 8 emit [char] | emit sync CO REPEAT ;
: ./   CO: BEGIN 8 emit [char] / emit sync CO REPEAT ;
CQINIT .- .\ .bar ./ GO

4 routines - and together they make up a spinner. The SYNC is there to flush the output channel - if not it waits for a CR. Pretty annoying if you're waiting for output.

So, the CO: simply registers the thing and continues. If we change all routines to:

." Init 1" cr CO:

We see:

Init 1
Init 2
Init 3
Init 4

<spinner>

Nothing special there. So - what's registered?

Init 1
CO: 94 (TOS)
Init 2
CO: 106 (TOS)
Init 3
CO: 118 (TOS)
Init 4
CO: 130 (TOS)
;CO 94 (TOS)
^H- CO 100 (TOS)
106 (TOS)
^H\ CO 112 (TOS)
118 (TOS)
^H| CO 124 (TOS)
130 (TOS)
^H/ CO 136 (TOS)
100 (TOS)
^H- CO 100 (TOS)
112 (TOS)
^H\ CO 112 (TOS)

Now this part is interesting:

      94| call                               71   CO:
    95| literal                             8
    96| emit                                0
    97| literal                            45
    98| emit                                0
    99| sync                                0
   100| call                               76   CO
   101| branch                             94
   102| exit                                0
   103| branch                            114   .\
   104| ."                                 23   Init 2
   105| cr                                  0
   106| call                               71   CO:
   107| literal                             8
   108| emit                                0
   109| literal                            92
   110| emit                                0
   111| sync                                0
   112| call                               76   CO
   113| branch                            106
   114| exit                                0
   115| branch                            126   .bar
   116| ."                                 30   Init 3
   117| cr                                  0
   118| call                               71   CO:
   119| literal                             8
   120| emit                                0
   121| literal                           124
   122| emit                                0
   123| sync                                0
   124| call                               76   CO
   125| branch                            118
   126| exit                                0
   127| branch                            138   ./
   128| ."                                 37   Init 4
   129| cr                                  0
   130| call                               71   CO:

All these are calls to CO: - so that's registered. The surprise was that ;CO was only called once! Obviously, it just starts the loop - and not GO as I assumed. I checked and yes, this will loop eternally as well:

CQINIT .- .\ .bar ./ ;CO

Also interesting - all routines have to be in an eternal loop. Remove that and this happens:

Init 1
CO: 94 (TOS)
Init 2
CO: 104 (TOS)
Init 3
CO: 114 (TOS)
Init 4
CO: 124 (TOS)
;CO 94 (TOS)
- CO 136 (TOS)
104 (TOS)
\ CO Executing; Word 78: Return stack empty

In short - it isn't even capable of completing one single loop. What is saved are initially the CO: addresses, but finally it ends up being a loop of CO addresses:

/ CO 136 (TOS)
100 (TOS)
- CO 100 (TOS)
112 (TOS)
\ CO 112 (TOS)
124 (TOS)
| CO 124 (TOS)
136 (TOS)
/ CO 136 (TOS)

etc.

Now - I assume that control is finally returned to the main loop if the routine exits - in other words, falls out of the loop and hits the EXIT word. And I assume that because that's how YIELD works.

The GO routine checks if the circular buffer is empty. If not, execution is resumed by ;CO (just like when the loop starts). That routine simply takes an address from the circular buffer and feeds it to the return stack - which starts the whole process. And that's all what this library does: it catches and stores return addresses - and retrieves and feeds them to the return stack (and let EXIT do his thing).

BTW, QUIT and INTERPRET have nothing to do with it - and I'm pretty sure of that since there is no such thing in 4tH - where the text interpreter is actually the compiler and the address interpreter is the virtual machine.

Hans Bezemer