FOR-EACH ... DO[ ... ]NEXT

[Gerry Jackson]

Complex loops with multiple WHILEs can be difficult to understand
particularly when returning to a program written some time ago. I wanted
a more readable control statement that expressed 'what' it did instead
of 'how' it did it. Also I wanted the ability to carry out a pipeline of
operations on a collection of data objects (array, linked list etc).

After some experimentation I ended up with this statement:
FOR-EACH <iterator> DO[ <pipeline operations> ]NEXT

The <iterator> is completely under the programmers control to provide
the next item in any type of data structure

Other requirements on pipeline operations:
- to be able to return to the <iterator> early i.e 'continue'
- to be able to exit the loop early i.e. 'break'
- (optional) possibly normal forth definitions

\ --------------------------------------

synonym for-each begin
synonym do[ while
synonym ]next repeat

: => ( f -- )
0 cs-pick postpone ?dup postpone 0= postpone until
; immediate

: pp=> ( -- ) postpone => ;

: p: : postpone [: ;
: ;p
postpone ;] postpone compile, postpone pp=> postpone ; immediate
; immediate

\ --------------------------------------

As can be seen this is just renaming BEGIN ... WHILE ... REPEAT for
readability with some rules that can be ignored or varied as needed by
the programmer.

- Pipeline operations can be defined using P: ... ;P or : ... ; the
difference being that P: incorporates the word => at the end to insert
the 'continue' control sequence. If normal colon definitions are used
and 'continue' is needed the word => must be used between operations

The rules to be used are:
- the <iterator> must return a non-zero value to continue the loop or a
zero to exit the loop.
- that pipeline operations must return a flag with possible values:
0 carry on to the next operation
-1 loop back to the iterator for the next iteration (continue)
+1 loop back to the iterator and exit the loop (break)

If neither 'continue' or 'break' are needed the FOR-EACH statement is
just a simple BEGIN ... WHILE ... REPEAT loop. Therefore normal colon
definitions can be used in the pipeline.

Using a simple/silly example display integers divisible by 6 that
Michael Gassanenko used to demonstrate his similar ... START ... EMERGE
... construct, we have:

: n+1 ( n1 n2 f1 -- n1 n2+1 f2 ) \ f2 true when n1<n2
drop 1+ 2dup >=
;

p: /2? ( n -- n f ) dup 2 mod ;p
p: /3? ( n -- n f ) dup 3 mod ;p
: .n ( n -- n ) dup . -1 ;

: /6? ( n1 n2 -- )
0 for-each n+1 do[ /2? /3? .n ]next 2drop
;

cr 50 0 /6? .s
\ displays 6 12 18 24 30 36 42 48

Alternatively using a 'break' in \2?

: n+1 ( n1 n2 f1 -- n1 n2+1 f2 ) \ f1 = +/-1
0> if 0 exit then 1+ true
;

p: /2? ( n1 n2 -- n1 n2 f ) 2dup < if 1 exit then dup 2 mod 0<> ;p
p: /3? ( n -- n f ) dup 3 mod 0<> ;p
: .n ( n -- n ) dup . -1 ;

: /6? ( n1 n2 -- )
for-each n+1 do[ /2? /3? .n ]next 2drop
;

cr 63 0 true /6? .s
\ displays 6 12 18 24 30 36 42 48 54 60

Of course imposing some rules means that some efficiency is lost
compared to WHILE loops but, as I use a desktop system, speed is never a
problem for me.

--
Gerry

[Brian Fox]

That's a very tidy way to do this. I was looking with lust at the Python
functions MAP REDUCE and FILTER. The key insight for me was that
they require an "initial value".

I play with a retro machine so memory is tight so I just used the
dictionary in a crude way to get a kind of dynamic array storage.
This allows the HOFs to output another array which can be
further processed. You can name an array with DATA:
or just use the result directly from the data stack.

Each array is stored as a CELL counted list of cells. SIZE converts
a counted array into an (addr,len) pair.

Here is what I came up with. (ignore the INCLUDE statements)
https://github.com/bfox9900/CAMEL99-ITC/blob/master/LIB.ITC/MAPCELLS.FTH

At the moment it only works for CELL data.
But of course the cells could be pointers. I just haven't gone farther.

I just tried it on GForth .73 and I broke the rules and didn't use CELL.
:-)
Now it works.
Might give you some ideas.

[Gerry Jackson]

On Friday, November 10, 2023 at 9:21:34 PM UTC, Brian Fox wrote:
> On Friday, November 10, 2023 at 12:45:47 PM UTC-5, Gerry Jackson wrote:

> > \ --------------------------------------
> >
> > synonym for-each begin
> > synonym do[ while
> > synonym ]next repeat
> >
> > : => ( f -- )
> > 0 cs-pick postpone ?dup postpone 0= postpone until
> > ; immediate
> >
> > : pp=> ( -- ) postpone => ;
> >
> > : p: : postpone [: ;
> > : ;p
> > postpone ;] postpone compile, postpone pp=> postpone ; immediate
> > ; immediate
> >
> > \ --------------------------------------
> >

> That's a very tidy way to do this. I was looking with lust at the
> Python functions MAP REDUCE and FILTER.

Such High Order Functions were one of the motivations for the FOR-EACH
statement

> The key insight for me was that they require an "initial value".

I'm not sure what you mean.

>
> I play with a retro machine so memory is tight so I just used the
> dictionary in a crude way to get a kind of dynamic array storage.
> This allows the HOFs to output another array which can be
> further processed. You can name an array with DATA:
> or just use the result directly from the data stack.
>
> Each array is stored as a CELL counted list of cells. SIZE converts
> a counted array into an (addr,len) pair.
>
> Here is what I came up with. (ignore the INCLUDE statements)
> https://github.com/bfox9900/CAMEL99-ITC/blob/master/LIB.ITC/MAPCELLS.FTH
>
> At the moment it only works for CELL data.
> But of course the cells could be pointers. I just haven't gone farther.
>
> I just tried it on GForth .73 and I broke the rules and didn't use CELL.
> :-)
> Now it works.
> Might give you some ideas.

Yes. Your work is with arrays only but, of course, can be extended to
other data types/structures. I am trying for a general solution that can
be applied to any type of data structure. For comparison here is a
definition of MAP with two examples of use on an integer array such as
used in your examples

: map ( ... iter-xt op-xt -- ... )
2>r for-each 2r@ drop execute do[ r@ execute ]next 2r> 2drop
;

\ The data aray
create a[] 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ,
10 constant a-len

\ The iterator, ad1 is end adress, ad2' the next array item address
:noname ( ad1 ad2 -- ad1 ad2' f )
cell+ 2dup u>
; constant next-item

\ The operation on each integer in the array
:noname ( ad -- ad ) dup @ dup * over ! ; constant squared

: init ( ad n -- ad+n' ad ) cells over + swap 1 cells - ;

: square[] ( ad n -- )
init next-item squared map 2drop
;

a[] a-len square[]

\ Display the array contents using MAP again

: .a[] init next-item [: ( ad -- ad ) dup @ . ;] map 2drop ;

cr a[] a-len .a[] cr

\ Displays
\ 1 4 9 16 25 36 49 64 81 100

--
Gerry

[none albert]

In article <uitpt1$qsjs$1...@dont-email.me>,

Gerry Jackson <do-no...@swldwa.uk> wrote:
>On Friday, November 10, 2023 at 9:21:34 PM UTC, Brian Fox wrote:
> > On Friday, November 10, 2023 at 12:45:47 PM UTC-5, Gerry Jackson wrote:
> > > \ --------------------------------------
> > >
> > > synonym for-each begin
> > > synonym do[ while
> > > synonym ]next repeat
> > >
> > > : => ( f -- )
> > > 0 cs-pick postpone ?dup postpone 0= postpone until
> > > ; immediate
> > >
> > > : pp=> ( -- ) postpone => ;
> > >
> > > : p: : postpone [: ;
> > > : ;p
> > > postpone ;] postpone compile, postpone pp=> postpone ; immediate
> > > ; immediate
> > >
> > > \ --------------------------------------
> > >
>
> > That's a very tidy way to do this. I was looking with lust at the
> > Python functions MAP REDUCE and FILTER.
>
>Such High Order Functions were one of the motivations for the FOR-EACH
>statement
>
> > The key insight for me was that they require an "initial value".
>
>I'm not sure what you mean.

If you want to apply + to a set you have to start with 0, OTOH
with * you must start with 1. If you want to print every word in
a wordlist you don't need it.

>
>Gerry
>
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat spinning. - the Wise from Antrim -

[minforth]

Gerry Jackson wrote:

> > That's a very tidy way to do this. I was looking with lust at the
> > Python functions MAP REDUCE and FILTER.

> Such High Order Functions were one of the motivations for the FOR-EACH
> statement

> > The key insight for me was that they require an "initial value".

> I'm not sure what you mean.

> >
> > I play with a retro machine so memory is tight so I just used the
> > dictionary in a crude way to get a kind of dynamic array storage.
> > This allows the HOFs to output another array which can be
> > further processed. You can name an array with DATA:
> > or just use the result directly from the data stack.

On machines with sufficient heap, it is worth implementing dynamic arrays.
I use an array stack and array values for this. They only contain a pointer
to the array structs in the heap. Then implementing HOFs for arrays is not
difficult (my arrays are mostly long 1-dimensional signal vectors with
thousands of elements).

Nice side effect: since strings are only character arrays, you get dynamic
strings for practically nothing.

[Gerry Jackson]

Yes, of course. I was looking for something more complicated. Thanks.

--
Gerry

[none albert]

In article <3bf703fe10a4ae72...@news.novabbs.com>,

I'm not sure what you gain here. On small machines you can hardly afford
setting aside space for a heap.

Groetjes Albert

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>Complex loops with multiple WHILEs can be difficult to understand
>particularly when returning to a program written some time ago.

Of course this makes me think of the extended case construct (present
in development Gforth).

Your variant is interesting, but it's not clear enough to me what it
can do. So let's see:

>Using a simple/silly example display integers divisible by 6 that
>Michael Gassanenko used to demonstrate his similar ... START ... EMERGE
>... construct, we have:
>
>: n+1 ( n1 n2 f1 -- n1 n2+1 f2 ) \ f2 true when n1<n2
> drop 1+ 2dup >=
>;
>
>p: /2? ( n -- n f ) dup 2 mod ;p
>p: /3? ( n -- n f ) dup 3 mod ;p
>: .n ( n -- n ) dup . -1 ;
>
>: /6? ( n1 n2 -- )
> 0 for-each n+1 do[ /2? /3? .n ]next 2drop
>;
>
>cr 50 0 /6? .s
>\ displays 6 12 18 24 30 36 42 48

Let's see how this turns out with the extended CASE:

: /6? ( limit start -- )
case ( limit n )
1+ 2dup < ?of 2drop endof
dup 2 mod ?of contof
dup 3 mod ?of contof
dup .
next-case ;

cr 50 0 /6?

This produces the expected output, and you can see how the parts of
the definition relate to the code in your variant.

Maybe there is an example where FOR-EACH ... DO[ ... ]NEXT is harder
to replace.

Given the number of cases where I have written nothing between the ?OF
and the following ENDOF or CONTOF, maybe a ?ENDOF or ?CONTOF
(equivalent to 0= ?OF ENDOF or 0= ?OF CONTOF) may be useful.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

[Anton Ertl]

an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>Given the number of cases where I have written nothing between the ?OF
>and the following ENDOF or CONTOF, maybe a ?ENDOF or ?CONTOF
>(equivalent to 0= ?OF ENDOF or 0= ?OF CONTOF) may be useful.

Or rather, without the "0="s

[Hans Bezemer]

On Friday, November 10, 2023 at 10:21:34 PM UTC+1, Brian Fox wrote:
> Now it works.
> Might give you some ideas.

Ideas like these?

: foreach ( 'f addr count -- )
cells bounds do
I @ over execute
loop drop ;

\ where : f ( n -- m )
: map ( 'f addr count -- )
cells bounds do
I @ over execute I !
loop drop ;

\ where : f ( st n -- st' )
: reduce ( st 'f addr count -- st' )
cells bounds do
I @ swap dup >r execute r>
loop drop ;

Hans Bezemer

[Gerry Jackson]

You'd need a different version of these for other collections e.g.
linked list. I was trying for a more general solution.

Its easy to write solutions like this for individual cases, its also
easy to make mistakes. Shouldn't the loops be ended with:
1 cells +loop

--
Gerry

[Doug Hoffman]

If you used an objects extension then the same syntax could work
for any type of collection.

-Doug

[minforth]

Gerry Jackson wrote:
> On 14/11/2023 18:31, Hans Bezemer wrote:
>> On Friday, November 10, 2023 at 10:21:34 PM UTC+1, Brian Fox wrote:
>> Ideas like these?
>>
>> : foreach ( 'f addr count -- )
>> cells bounds do
>> I @ over execute
>> loop drop ;
>>
>> where : f ( n -- m )
>> : map ( 'f addr count -- )
>> cells bounds do
>> I @ over execute I !
>> loop drop ;
>>
>> where : f ( st n -- st' )
>> : reduce ( st 'f addr count -- st' )
>> cells bounds do
>> I @ swap dup >r execute r>
>> loop drop ;
>>

> Its easy to write solutions like this for individual cases, its also
> easy to make mistakes. Shouldn't the loops be ended with:
> 1 cells +loop

Things start getting funny when you want AVX2 instructions under the hood. ;-)
For individual vector type cases of course.

A more general approach is nevertheless useful for testing and prototyping.

[Helmut Eller]

On Fri, Nov 10 2023, Gerry Jackson wrote:

> Also I wanted the ability to carry out a pipeline of operations on a
> collection of data objects (array, linked list etc).

I wanted to have something like Rust's iterator library and played
around with the code below. With this, the divisible-by-6 example could
be written as:

: /2? ( n -- flag ) 2 mod 0= ;
: /3? ( n -- flag ) 3 mod 0= ;

: /6? ( start end -- )
[range-iterator]
[ ' /2? ] [filter]
[ ' /3? ] [filter]
[ ' . ] [for-each]
;

This is arguably very close to the Rust expression:

(start..end)
.filter(|u| u % 2 == 0)
.filter(|u| u % 3 == 0)
.for_each(|u| print!("{u} "));

Unfortunately, my code is much more complicated that yours. On the plus
side, I'm trying to mimic Rust's operators and naming, which saves me
from re-inventing wheels and perhaps helps to communicate the intention
behind the operators.

Helmut


\ iter.fth --- Rust inspired iterators
\
\ Iterators are built around a function of type:
\
\ next ( state -- state' item flag )
\
\ i.e. the NEXT function computes the next state of the iterator and
\ yields ITEMs. FLAG is true if the item is valid and false if the
\ iterator is finished (in that case the item is invalid).
\
\ This code tries to be “generic” in the sense that STATE and ITEM can
\ be more than a single stack item. E.g. to iterate over the lines in
\ a file, ITEM can be a pair C-ADDR U to represent the string. The
\ size of STATE and ITEM must be known at compile time.
\
\ An example for client code is:
\
\ : foo ( ) 0 10 [range-iterator] [ ' . ] [for-each] ;
\ foo
\
\ As in Rust, a “range” is an interval of integers, and the example
\ prints the integers from 0 up to 9.
\
\ At compile time, [RANGE-ITERATOR] puts a data structure on the stack
\ that describes, among other things, the size of STATE and NEXT.
\ [FOR-EACH] uses this structure to assemble the loop. In this
\ example the produced code looks like this:
\
\ : foo 0 10 BEGIN %range-next 0<> WHILE . REPEAT drop 2drop ;
\
\ Here, %RANGE-NEXT is a specific next function that moves the
\ iterator from one state to the next. The STATE for a range iterator
\ requires two stack slots and the ITEM one slot. [FOR-EACH] also
\ assumes that the xt (produced by the [ ' . ] part in the example)
\ has type ( ITEM -- ).
\

forth-wordlist wordlist 2 set-order definitions

\ Define Gforth's r-r-bracket if needed
[undefined] ]] [if]
: refilling-parse-name ( -- old->in c-addr u )
begin
>in @ parse-name dup 0= while
2drop drop refill 0= -39 and throw
repeat
;

: ]] ( -- )
begin
refilling-parse-name s" [[" compare while
>in ! POSTPONE postpone
repeat
drop
; immediate
[then]

0
1 cells +field it.nargs \ number of args for the constructor
1 cells +field it.nstate \ number of stack slots for iterator state
1 cells +field it.nitem \ number of stack slots per yielded item
1 cells +field it.init \ xt: ( it -- )
1 cells +field it.next \ xt: ( it -- )
1 cells +field it.drop \ xt: ( it -- )
1 cells +field it.next-back \ xt: ( it -- )
constant /iter-type

: [for-each] {: it xt -- :}
it dup it.init @ execute
]]
begin
[[ it dup it.next @ execute ]] 0<> while
[[ xt compile, ]]
repeat
[[ it it.nitem @ 0 ?do postpone drop loop ]]
[[ it dup it.drop @ execute ]]
[[
; immediate

: init-it ( nargs nstate nitem init next drop next-back a-addr -- it )
>r
r@ it.next-back !
r@ it.drop !
r@ it.next !
r@ it.init !
r@ it.nitem !
r@ it.nstate !
r@ it.nargs !
r>
;

: make-it ( nargs nstate nitem init next drop next-back -- it )
/iter-type allocate throw init-it
;

: %range-next ( lo hi -- lo' hi lo flag )
2dup = if 0 false else over >r swap 1+ swap r> true then
;

: %range-next-back ( lo hi -- lo hi' hi' flag )
2dup = if 0 false else 1- dup true then
;

: range-next ( it -- ) drop postpone %range-next ;
: range-next-back ( it -- ) drop postpone %range-next-back ;
: range-drop ( it -- ) drop postpone 2drop ;

: todo ( -- ) true abort" not yet implemented" ;
: identity ( -- ) ;

: [range-iterator] ( -- it )
2
2
1
['] drop
['] range-next
['] range-drop
['] range-next-back
make-it
; immediate

/iter-type
1 cells +field filter.it
1 cells +field filter.xt
constant /filter

: filter-init ( it -- ) filter.it @ dup it.init @ execute ;

: %then ( orig -- ) postpone then ; immediate

: filter-next {: it -- :}
]]
begin
[[ it filter.it @ dup it.next @ execute ]]
0= if false ahead [[ 1 cs-roll ]] then
dup [[ it filter.xt @ compile, ]] if true ahead [[ 1 cs-roll ]] then
drop
[[ 2 cs-roll ]]
again %then %then
[[
;

: filter-drop ( it -- ) filter.it @ dup it.drop @ execute ;

: make-filter ( it xt -- it2 )
/filter allocate throw >r
r@ filter.xt !
r@ filter.it !
r@ filter.it @ it.nstate @
r@ filter.it @ it.nstate @
r@ filter.it @ it.nitem @
['] filter-init
['] filter-next
['] filter-drop
['] todo
r> init-it
;

: [filter] ( it xt -- it2 ) make-filter ; immediate

/iter-type
1 cells +field map.it
1 cells +field map.xt
constant /map

: map-next {: it -- :}
it map.it @ dup it.next @ execute
]]
if
[[ it map.xt @ compile, ]]
true
else
[[
it map.it @ it.nitem @ it it.nitem @ -
dup 0< if
negate 0 ?do 0 postpone literal loop
else
0 ?do postpone drop loop
then
]]
false
then
[[
;

: map-drop ( it -- ) map.it @ dup it.drop @ execute ;
: map-init ( it -- ) map.it @ dup it.init @ execute ;

: make-map ( it u xt -- it2 )
/map allocate throw >r
r@ map.xt !
r@ it.nitem !
r@ map.it !
r@ map.it @ it.nstate @
dup
r@ it.nitem @
['] map-init
['] map-next
['] map-drop
['] todo
r> init-it
;

: [map] ( it xt -- it2 ) make-map ; immediate

/iter-type
1 cells +field reverse.it
constant /reverse

: reverse-init ( it -- ) reverse.it @ dup it.init @ execute ;
: reverse-next ( it -- ) reverse.it @ dup it.next-back @ execute ;
: reverse-drop ( it -- ) reverse.it @ dup it.drop @ execute ;
: reverse-next-back ( it -- ) reverse.it @ dup it.next @ execute ;

: make-reverse ( it -- it2 )
/reverse allocate throw >r
r@ reverse.it !
0
r@ reverse.it @ it.nstate @
r@ reverse.it @ it.nitem @
['] reverse-init
['] reverse-next
['] reverse-drop
['] reverse-next-back
r> init-it
;

: [reverse] ( it -- it2 ) make-reverse ; immediate

/iter-type
1 cells +field zip.it1
1 cells +field zip.it2
constant /zip

: zip-init ( it -- )
dup zip.it1 @ dup it.init @ execute
zip.it2 @ dup it.init @ execute
;

\ Generate code to rotate SIZE items at depth DEPTH to the top of the stack,
\ i.e. the stack effect should be: ( size×x depth×x -- depth×x size×x )
\
\ 1 1 NROLL corresponds to SWAP,
\ 2 2 NROLL to 2SWAP and
\ 1 2 NROLL to ROT.
: nroll ( size depth -- )
{: s d :}
\ s d 1 1 d= if postpone swap exit then
\ s d 2 2 d= if postpone 2swap exit then
\ s d + 1- 2 = if s 0 ?do postpone rot loop exit then
\ d 0= if exit then
s 0 ?do
s d + 1- postpone literal postpone roll
loop
;

: zip-next ( it -- )
dup zip.it1 @ swap zip.it2 @ {: it1 it2 :}
it1 it.nstate @ it2 it.nstate @ nroll
it1 dup it.next @ execute
]] if
[[
it2 it.nstate @ it1 it.nstate @ it1 it.nitem @ + nroll
it2 dup it.next @ execute
it2 it.nstate @ it2 it.nitem @ 1+ nroll
it1 it.nitem @ it2 it.nitem @ 1 + + it2 it.nstate @ nroll
]]
else
[[
it2 it.nstate @ it1 it.nitem @ nroll
it1 it.nstate @ it1 it.nitem @ + it2 it.nstate @ nroll
it2 it.nitem @ 0 ?do 0 postpone literal loop
]]
false
then
[[
;

: zip-drop ( it -- )
dup zip.it2 @ dup it.drop @ execute
zip.it1 @ dup it.drop @ execute
;

: make-zip ( it1 it2 -- it3 )
/zip allocate throw >r
r@ zip.it2 !
r@ zip.it1 !
r@ zip.it1 @ it.nstate @ r@ zip.it2 @ it.nstate @ +
dup
r@ zip.it1 @ it.nitem @ r@ zip.it2 @ it.nitem @ +
['] zip-init
['] zip-next
['] zip-drop
['] todo
r> init-it
;

: [zip] ( it1 it2 -- it3 ) make-zip ; immediate

/iter-type
1 cells +field enumerate.it
constant /enumerate

: enumerate-init ( it -- )
enumerate.it @ dup it.init @ execute
0 postpone literal
;

: enumerate-next ( it -- )
enumerate.it @ {: it :}
it it.nstate @ 1 nroll
it dup it.next @ execute
1 it it.nstate @ it it.nitem @ 1+ + nroll
]] tuck 1+ [[
it it.nitem @ 2 + 1 nroll
;

: enumerate-drop ( it -- )
postpone drop
enumerate.it @ dup it.drop @ execute
;

: make-enumerate ( it -- it2 )
/enumerate allocate throw >r
r@ enumerate.it !
r@ enumerate.it @ it.nstate @
dup 1+
r@ enumerate.it @ it.nitem @ 1+
['] enumerate-init
['] enumerate-next
['] enumerate-drop
['] todo
r> init-it
;

: [enumerate] ( it -- it2 ) make-enumerate ; immediate

: [fold] {: it u xt -- :}
it it.nargs @ u nroll
it dup it.init @ execute
]] begin
[[ it dup it.next @ execute ]] while
[[
u it it.nstate @ it it.nitem @ + nroll
it it.nitem @ u nroll
xt compile,
it it.nstate @ u nroll
]]
repeat
[[
it it.nitem @ 0 ?do postpone drop loop
it dup it.drop @ execute
; immediate

0
1 cells +field line-iter.file
2 cells +field line-iter.buffer
constant /line-iter

: make-line-iter ( file buffer-size -- &line-iter )
/line-iter allocate throw >r
dup allocate throw swap r@ line-iter.buffer 2!
r@ line-iter.file !
r>
;

/iter-type
constant /lines

: %line-iter-next ( &line-iter -- &line-iter c-addr u flag )
dup >r
r@ line-iter.buffer 2@
over swap r> line-iter.file @ read-line
throw
;

: %line-iter-drop ( &line-iter -- )
>r
r@ line-iter.buffer 2@ drop free throw
r@ line-iter.file @ close-file throw
r> drop
;

: lines-init ( it -- ) drop postpone make-line-iter ;
: lines-next ( it -- ) drop postpone %line-iter-next ;
: lines-drop ( it -- ) drop postpone %line-iter-drop ;

: [lines-iterator] ( -- it )
2
1
2
['] lines-init
['] lines-next
['] lines-drop
['] todo
make-it
; immediate

get-current previous definitions constant iter-wordlist

\ tests
forth-wordlist iter-wordlist wordlist 3 set-order definitions

: #u ( u -- ) s>d #s 2drop bl hold ;

: test-range-iterator ( -- )
<#
0 10 [range-iterator]
[ ' #u ] [for-each]
0 0 #>
s" 9 8 7 6 5 4 3 2 1 0" compare 0<> abort" test-range-iterator failed"
;

\ see test-range-iterator

: test-reverse ( -- )
<#
0 10 [range-iterator] [reverse]
[ ' #u ] [for-each]
0 0 #>
s" 0 1 2 3 4 5 6 7 8 9" compare 0<> abort" test-reverse failed"
;

: .pair ( u1 u2 -- ) ." (" swap . 0 .r ." ) " ;

: #pair ( u1 u2 -- )
s" )" holds swap s>d #s 2drop bl hold s>d #s 2drop s" (" holds
;

: test-zip ( -- )
<#
0 10 0 4
[range-iterator] [reverse]
[range-iterator]
[zip]
[ ' #pair ] [for-each]
0 0 #>
s" (3 6) (2 7) (1 8) (0 9)" compare 0<> abort" test-zip failed"
;

: test-fold ( -- )
0 10 [range-iterator]
1234 0 [ 2 ' + ] [fold]
1234 9 10 * 2/ d= 0= abort" test-fold failed"
;

\ see test-fold

: test-map ( -- )
<#
0 5 [range-iterator]
[ 2 ' dup ] [map]
[ ' #pair ] [for-each]
0 0 #>
s" (4 4) (3 3) (2 2) (1 1) (0 0)" compare 0<> abort" test-map failed"
;

\ see test-map

: test-enumerate ( -- )
<#
10 13 [range-iterator]
[enumerate]
[ ' #pair ] [for-each]
0 0 #>
\ 2dup type
s" (2 12) (1 11) (0 10)" compare 0<> abort" test-enumerate failed"
;

\ see test-enumerate

: .line ( c-addr u line# -- ) 2 .r ." : " type ;

: test-lines ( -- )
s" /etc/motd" r/o open-file throw 256 [lines-iterator]
[enumerate]
[ 0 ' .line ] [map]
[ ' cr ] [for-each]
;

\ see test-lines

: /2? ( n -- flag ) 2 mod 0= ;
: /3? ( n -- flag ) 3 mod 0= ;

: /6? ( start end -- )
[range-iterator]
[ ' /2? ] [filter]
[ ' /3? ] [filter]
[ ' . ] [for-each]
;

\ see /6?

: test-/6? ( -- )
<#
0 50 [range-iterator]
[ ' /2? ] [filter]
[ ' /3? ] [filter]
[ ' #u ] [for-each]
0 0 #>
s" 48 42 36 30 24 18 12 6 0" compare 0<> abort" test-/6? failed"
;

: run-tests ( -- )
test-range-iterator
test-reverse
test-zip
test-fold
test-map
test-enumerate
test-lines
test-/6?
depth 0<> abort" non-zero depth"
." tests passed" cr
;

run-tests
bye

[Gerry Jackson]

Your extended case is very versatile so I'm not surprised it can do the
same function. However you are missing the point of what I'm trying to
achieve. This is to have a versatile looping structure that is more
readable, one that hides the nuts and bolts of the code that implements
the loop. A solution that expresses what has to be done, not one that
dictates how it will be done i.e. a more readable solution. I think your
solution doesn't meet that readability criterion, not that such a simple
example is difficult to understand.

>
> Maybe there is an example where FOR-EACH ... DO[ ... ]NEXT is harder
> to replace.

Probably not. If you want a challenge an interesting question is whether
it is simpler than my attempt to implement the FOR-EACH construct, or
similar, using your extended CASE.

>
> Given the number of cases where I have written nothing between the ?OF
> and the following ENDOF or CONTOF, maybe a ?ENDOF or ?CONTOF
> (equivalent to 0= ?OF ENDOF or 0= ?OF CONTOF) may be useful.
>

Couldn't ?OF parse the next word to see if it is ?ENDOF or ?CONTOF and
generate the appropriate code if so?

--
Gerry

[Gerry Jackson]

On Wednesday, November 15, 2023 at 10:35:37 AM UTC, Helmut Eller wrote:
> On Fri, Nov 10 2023, Gerry Jackson wrote:
>
> > Also I wanted the ability to carry out a pipeline of operations on a
> > collection of data objects (array, linked list etc).

> I wanted to have something like Rust's iterator library and played
> around with the code below. With this, the divisible-by-6 example could
> be written as:
>
> : /2? ( n -- flag ) 2 mod 0= ;
> : /3? ( n -- flag ) 3 mod 0= ;
>
> : /6? ( start end -- )
> [range-iterator]
> [ ' /2? ] [filter]
> [ ' /3? ] [filter]
> [ ' . ] [for-each]
> ;
>
> This is arguably very close to the Rust expression:
>
> (start..end)
> .filter(|u| u % 2 == 0)
> .filter(|u| u % 3 == 0)
> .for_each(|u| print!("{u} "));
>
> Unfortunately, my code is much more complicated that yours. On the plus
> side, I'm trying to mimic Rust's operators and naming, which saves me
> from re-inventing wheels and perhaps helps to communicate the intention
> behind the operators.
>
> Helmut
>
>

Wow, you've done a lot more work on this than I have.
It's an excellent idea to mimic another language you're familiar with,
I've never looked at Rust.

Thank you for posting all your code, I shall have a more detailed
look at it. It's nice to know that someone else is attempting to
do (or has done) something similar to what I am attempting.

Gerry

\ iter.fth --- Rust inspired iterators

[ Code for the above snipped]

[Gerry Jackson]

On 15/11/2023 16:33, Gerry Jackson wrote:
>
> Couldn't ?OF parse the next word to see if it is ?ENDOF or ?CONTOF and
> generate the appropriate code if so?
>

Sorry I meant ENDOF or CONTOF

--
Gerry

[Anton Ertl]

The reader needs to be familiar with FOR-EACH ... DO[ ... ]NEXT and

P: ... ;P to make this more readable. The question is if this will be
used frequently enough to make readers familiar with these words.
People have already argued against the extended CASE because of
unfamiliarity.

OTOH, if there are enough uses of the extended CASE for the
filter-loop pattern, one might also find it readable (and recognize
that it is a filter loop) when expressed using the extended CASE.

>> Given the number of cases where I have written nothing between the ?OF
>> and the following ENDOF or CONTOF, maybe a ?ENDOF or ?CONTOF
>> (equivalent to 0= ?OF ENDOF or 0= ?OF CONTOF) may be useful.
>>
>
>Couldn't ?OF parse the next word to see if it is ?ENDOF or ?CONTOF and
>generate the appropriate code if so?

The idea behind ?ENDOF and ?CONTOF is to increase readability by
making the code shorter, not optimization. So it might be:

: /6? ( limit start -- )
case

1+ 2dup < ?endof
dup 2 mod ?contof dup 3 mod ?contof dup . next-case
2drop ;

As for optimizing ?OF CONTOF, I would not do it with a parsing word,
because of bad experiences with parsing words. I would probably do it
by optimizing a conditional branch (?OF) that just jumps over an
unconditional branch (ENDOF): just invert the sense of the conditional
branch and let it's target be that of the unconditional branch; leave
the unconditional branch away.

[none albert]

In article <2023Nov1...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

>The idea behind ?ENDOF and ?CONTOF is to increase readability by
>making the code shorter, not optimization. So it might be:
>
>: /6? ( limit start -- )
> case
> 1+ 2dup < ?endof
> dup 2 mod ?contof dup 3 mod ?contof dup . next-case
> 2drop ;
>
>As for optimizing ?OF CONTOF, I would not do it with a parsing word,
>because of bad experiences with parsing words. I would probably do it
>by optimizing a conditional branch (?OF) that just jumps over an
>unconditional branch (ENDOF): just invert the sense of the conditional
>branch and let it's target be that of the unconditional branch; leave
>the unconditional branch away.

You introduced CONDS .. THENS .
I have used this often. There is an advantage over CASE that it
is just a familiar nested THEN .
A mistake in a control structure is more cumbersome
than a stack error that is easily spotted by an interpretitive test.

Since that time I never used CASE, that I find hard to use.
If parsing words are added, that makes it worse.
CASE doesn't save on words.

>
>- anton

[Hans Bezemer]

On Wednesday, November 15, 2023 at 10:09:48 AM UTC+1, Gerry Jackson wrote:
> Its easy to write solutions like this for individual cases, its also
> easy to make mistakes. Shouldn't the loops be ended with:
> 1 cells +loop

True. But this code was before 4tH could optimize 1 CELLS +LOOP away. Since
in 4tH cells have their own segment (and hence the number of "units" per cell is
always 1) I got myself a bit of performance by writing it that way.

TL;DR - You're completely right in an ANS Forth way.

Hans Bezemer

[jes sutarman]

UANG618
👉 Klik Disini Daftar Slot Gacor ❱❱ https://bit.ly/DEPO-5000 📌
👉 Klik Disini Daftar Bonus New Member ❱❱ https://bit.ly/DEPO-5000 📌
👉 Klik DisiniDaftar Slot Garansi Kekalahan ❱❱https://bit.ly/DEPO-5000 📌
🥇Bonus New Member 50%
🥈Bonus Next Deposit 15 % Klaim 1x Dalam 1 Hari
🥉Bonus Next Deposit 10 % Klaim 2x Dalam 1 Hari
🥉Bonus Next Deposit 5 % Klaim Sepuasnya
🥉Bonus Anti Rungkad 20%
🥉Promo Begadang Up Rp.300.000
🥉Bonus Garansi Kekalahan 25 %
🥉Promo Birthday Bonanza
🥉Bonus Mingguan/ Cashback 5%
🥉Bonus Mingguan/Rolingan 0.5%
🥉Event Gates Of Olympus
🥉Event Bonanza Boom
🥉Event Koi Gate

🔥SEGERA DAFTAR DI UANG 618 BO AMAN & TERPERCAYA, BURUAN BESTIEE ‼️🔥

[Gerry Jackson]

I think using familiarity is a weak argument. People can become familiar
with virtually anything if they use it often enough.

I think the extended case is very versatile but ISTM that it has a noisy
syntax and from one point of view is inconsistent. For example if we
take the original CASE statement, in its basic use:
(n1 n2) OF ... ENDOF i.e. n1=n2 can be regarded as success.

In the extended CASE, it can be more like a pipeline where:
( f ) ?OF ... ENDOF leaves the pipeline, there f=TRUE can be regarded as
failure. OTOH the same thing in other uses it could be regarded as success.

Anyway having said that we can probably ignore that as its flexibility
is extremely useful. But I would like to use it in a more readable form
such as the aforementioned FOR-EACH etc.

I've been experimenting with it to implement the FOR-EACH construct and
it is very easy. I started with GForth's ANS compatibilty version and
found that it isn't compatible with my system because my system uses a
separate control stack which is permitted in ANS Forth. The GForth
compatible version uses DEPTH as a way of providing the correct number
of POSTPONE THENs to resolve the ENDOFs. Hence the failure on my system
as DEPTH is unlikely to change. Then I vaguely remembered that I had
posted some similar code years ago on c.l.f and managed to track it down at:
https://groups.google.com/g/comp.lang.forth/c/64GKthsYVFs/m/1QTLZCyiHCUJ

Using that the implemention is straightforward.

: for-each ( RT: -- ) postpone case ; immediate
: do[ ( RT: f -- ) postpone ?break ; immediate
: ]next ( RT: x -- ) postpone dup postpone next-case ; immediate
: ?next ( RT: f -- ) postpone ?of postpone contof ; immediate

?BREAK is defined in the post at the above link and is equivalent to:
?OF ENDOF

if we want to us a similar syntax to Helmut Eller's excellent post, we
could define:

synonym filter ?next

The example used previously is then:

: n+1 ( n1 n2 -- n1 n2+1 f ) \ f true when n1<n2
1+ 2dup <
;

: /2? ( n -- n f ) dup 2 mod ;

: /3? ( n -- n f ) dup 3 mod ;

: .n ( n -- n ) dup . ;

: /6? ( n1 n2 -- )

for-each n+1 do[ /2? ?next /3? ?next .n ]next 2drop
;

50 0 /6? .s

Incidentally I gave up having P: ... ;P as a bad idea.

--
Gerry

[Helmut Eller]

On Sun, Nov 19 2023, Gerry Jackson wrote:
> Using that the implemention is straightforward.
>
> : for-each ( RT: -- ) postpone case ; immediate
> : do[ ( RT: f -- ) postpone ?break ; immediate
> : ]next ( RT: x -- ) postpone dup postpone next-case ; immediate

I think the DUP is too much as NEXT-CASE, unlike ENDCASE, doesn't drop.
Either way, the stack comment doesn't seem to match the code.

> : ?next ( RT: f -- ) postpone ?of postpone contof ; immediate
>
> ?BREAK is defined in the post at the above link and is equivalent to:
> ?OF ENDOF

I.e.:

: ?break ( RT: f -- ) postpone ?of postpone endof ; immediate

> if we want to us a similar syntax to Helmut Eller's excellent post, we
> could define:
>
> synonym filter ?next

It would probably be inverted:

: filter ( RT: f -- ) postpone 0= postpone ?next ; immediate

> The example used previously is then:
>
> : n+1 ( n1 n2 -- n1 n2+1 f ) \ f true when n1<n2
> 1+ 2dup <
> ;
>
> : /2? ( n -- n f ) dup 2 mod ;
> : /3? ( n -- n f ) dup 3 mod ;
> : .n ( n -- n ) dup . ;
>
> : /6? ( n1 n2 -- )
> for-each n+1 do[ /2? ?next /3? ?next .n ]next 2drop
> ;
>
> 50 0 /6? .s

Mathematically speaking, zero is divisible by 6. So I think 0 should be
printed too. This could probably done with a different definition for
n+1. Also the naming is a bit curious: isn't /2? supposed to read as
"is divisible by 2"? But the implementation returns the inverse: true if
it is not divisible.

> Incidentally I gave up having P: ... ;P as a bad idea.

It think there was something useful to the idea that a helper can return
three values: -1, 0, +1. In particular it could be useful to separate
the "break because we're finished" case from the "break because we've
encountered an error" situation.

Helmut

[Ruvim]

I just realized that this "case" is actually a loop, and it's similar to
my fancy curly control flow structures [1], which misses an instruction
to premature continue execution from the beginning of the loop.

So I added now this instruction, and the word "/6?" can be defined as
follows:

: /6? ( limit start -- )

repeat{ ( limit n )
1+ 2dup < if-break{ 2drop }
dup 2 mod if-cont{}
dup 3 mod if-cont{}
dup .
}repeat
;


NB: any ending instruction can be written in a long or short "}" form.
So "if-cont{}" can be written as "if-cont{ ... }"
and "if-cont{ ... }if-cont", with a non-empty body, which is executed
before continue from the beginning of the loop.




[1]
https://github.com/ruv/forth-on-forth/blob/master/lib/control-flow-curly.fth
https://github.com/ruv/forth-on-forth/blob/master/lib/control-flow-curly.test.fth


--
Ruvim

[dxf]

On 21/11/2023 11:02 am, Ruvim wrote:
>
> ... the word "/6?" can be defined as follows:

>
>   : /6? ( limit start -- )
>     repeat{ ( limit n )
>       1+ 2dup < if-break{ 2drop }
>       dup 2 mod if-cont{}
>       dup 3 mod if-cont{}
>       dup .
>     }repeat
>   ;

: ?six ( n -- n )
dup 2 mod if exit then
dup 3 mod if exit then
dup .
;

: /6? ( limit start -- )

begin
1+ 2dup < 0=
while
?six
repeat 2drop
;

[Ruvim]

Your variant is longer by 3 lines out of 8, which is 3/8 = 38%

Another rationale is the same as for quotations: sometime you don't want
to introduce a new word that is used only once.


--
Ruvim

[dxf]

Named forth subroutines were not intended as a punishment.

[Jan Coombs]

On Tue, 21 Nov 2023 20:05:56 +1100
dxf <dxf...@gmail.com> wrote:

> > Your variant is longer by 3 lines out of 8, which is 3/8 = 38%

1 compiling token more in ~21 is under 5% in code space.

More text may make the intention clearer, and to my mind also does so.

However Ruvim, thanks for new ideas, have bookmarked to read again.

Jan Coombs
--

[Gerry Jackson]

Sorry about the delay in replying - family took priority!

On 20/11/2023 16:05, Helmut Eller wrote:
> On Sun, Nov 19 2023, Gerry Jackson wrote:
>> Using that the implemention is straightforward.
>>
>> : for-each ( RT: -- ) postpone case ; immediate
>> : do[ ( RT: f -- ) postpone ?break ; immediate
>> : ]next ( RT: x -- ) postpone dup postpone next-case ; immediate
>
> I think the DUP is too much as NEXT-CASE, unlike ENDCASE, doesn't drop.
> Either way, the stack comment doesn't seem to match the code.

Sorry you're right, I was under the mistaken impression that the GForth
NEXT-CASE included a DROP.

>
>> : ?next ( RT: f -- ) postpone ?of postpone contof ; immediate
>>
>> ?BREAK is defined in the post at the above link and is equivalent to:
>> ?OF ENDOF
>
> I.e.:
>
> : ?break ( RT: f -- ) postpone ?of postpone endof ; immediate

In the link to CASE extensions I defined ?BREAK a bit more efficiently as:

\ ?break is equivalent to an empty ?OF ENDOF

: cs-swap ( orig1/dest1 orig2/dest2 -- orig2/dest2 orig1/dest1 )
1 cs-roll
;

: ?break ( C: dest #of -- orig dest #of+1 ) \ run-time ( f -- )
>r postpone 0= postpone if cs-swap r> 1+
; immediate

The #OF is a compile time count of the number of THENs required to
resolve the number of origs due to ?BREAKs and ENDOFs

>
>> if we want to us a similar syntax to Helmut Eller's excellent post, we
>> could define:
>>
>> synonym filter ?next
>
> It would probably be inverted:
>
> : filter ( RT: f -- ) postpone 0= postpone ?next ; immediate

ok

>
>> The example used previously is then:
>>
>> : n+1 ( n1 n2 -- n1 n2+1 f ) \ f true when n1<n2
>> 1+ 2dup <
>> ;
>>
>> : /2? ( n -- n f ) dup 2 mod ;
>> : /3? ( n -- n f ) dup 3 mod ;
>> : .n ( n -- n ) dup . ;
>>
>> : /6? ( n1 n2 -- )
>> for-each n+1 do[ /2? ?next /3? ?next .n ]next 2drop
>> ;
>>
>> 50 0 /6? .s
>
> Mathematically speaking, zero is divisible by 6. So I think 0 should be
> printed too. This could probably done with a different definition for
> n+1.

Yes the example is a bit misleading because of the 1+ at the start of
the iterator n+1 so the code is testing 1 to n2. A better way is to put
1- in the definition for /6?

e.g. 50 -10 /6? displays -6 0 6 12 18 24 30 36 42 48

> Also the naming is a bit curious: isn't /2? supposed to read as
> "is divisible by 2"? But the implementation returns the inverse: true if
> it is not divisible.

Yes that is so, I started using a previous example from years ago and
never really thought about that aspect. Probably I should have named the
words mod2? and mod3? instead of /2? and /3?

>
>> Incidentally I gave up having P: ... ;P as a bad idea.
>
> It think there was something useful to the idea that a helper can return
> three values: -1, 0, +1. In particular it could be useful to separate
> the "break because we're finished" case from the "break because we've
> encountered an error" situation.

Yes that is true. But the "break because we've encountered an error"
situation can be handled by the case, that detected the error, THROWing
an exception or aborting instead of generating a +1. Also using a +1
wouldn't distinguish between which of 2 or more cases that could
generate a +1, but that could be handled by the different cases
generating +1, +2, ... etc as any positive number would do.

>
> Helmut

The revised definitions become:
: for-each ( RT: -- ) postpone case ; immediate \ RT: is run time

: do[ ( RT: f -- ) postpone ?break ; immediate

: ]next ( RT: x -- ) postpone next-case ; immediate

: ?next ( RT: f -- ) postpone ?of postpone contof ; immediate

\ Example

: n+1 ( n1 n2 -- n1 n2+1 f ) \ f true when n1<n2
1+ 2dup <
;

: mod2? ( n -- n f ) dup 2 mod ;
: mod3? ( n -- n f ) dup 3 mod ;

: .n ( n -- n ) dup . ;

: /6? ( n1 n2 -- )

1- for-each n+1 do[ mod2? ?next mod3? ?next .n ]next 2drop
;

50 0 /6? .s \ displays 0 6 12 18 24 30 36 42 48

Thanks for your comments.

--
Gerry

[Gerry Jackson]

On 28/11/2023 11:50, Gerry Jackson wrote:
> The revised definitions become:
> : for-each ( RT: -- )  postpone case  ; immediate \ RT: is run time
> : do[ ( RT: f -- )  postpone ?break  ; immediate
> : ]next ( RT: x -- )  postpone next-case  ; immediate
> : ?next  ( RT: f -- )  postpone ?of postpone contof  ; immediate
>
> \ Example
> : n+1  ( n1 n2 -- n1 n2+1 f ) \ f true when n1<n2
>    1+ 2dup <
> ;
>
> : mod2?  ( n -- n f )  dup 2 mod  ;
> : mod3?  ( n -- n f )  dup 3 mod  ;
> : .n   ( n -- n )  dup .  ;
>
> : /6?  ( n1 n2  -- )
>    1- for-each n+1 do[ mod2? ?next mod3? ?next .n ]next 2drop
> ;
>
> 50 0 /6? .s \ displays 0 6 12 18 24 30 36 42 48
>
> Thanks for your comments.
>

I've been refining the implementation of the FOR-EACH statement and have
re-implented it based on the BEGIN ... WHILE ... REPEAT loop instead of
the GForth extended CASE. Differences from the previous version are:

- the iterator can ignore whether the pipeline operations CONTINUE or
BREAK as they jump back to a conditional jump just before the iterator

- the ]NEXT code jumps back to the iterator itself

- the pipeline operator has changed from => to |> which is used in other
languages

These make the control flow rather convoluted.

The FOR-EACH definitions are:

: cs-drop-dest ( CS: dest -- )
postpone ahead 1 cs-roll postpone again postpone then
;

: for-each
postpone ahead postpone begin postpone 0< postpone if
postpone begin 3 cs-roll postpone then
; immediate

synonym do[ while [defined] [SwiftForth] [if] immediate [then]

: ]next
postpone repeat postpone then cs-drop-dest
; immediate

\ The pipeline operator
: |> ( f -- ) 3 cs-pick postpone ?dup postpone 0= postpone until ;
immediate

This ran the previous examples on 6 different Forth systems correctly
but with the following examples 2 of the 6 Forths failed.

Example A - MAP without locals
: map ( ... iter-xt op-xt -- ... )
2>r for-each 2r@ drop execute do[ r@ execute ]next 2r> 2drop
;

Example B - MAP with locals for the xts
: map ( ... iter-xt op-xt -- ... )
{: it-xt op-xt :} for-each it-xt execute do[ op-xt execute ]next
;

Testing these alternatives with the following data and definitions that
simply square the items in an array:

\ ------------------------------------
create a[] 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ,
10 constant a-len

:noname ( ad1 ad2 -- ad1 ad2' f ) \ ad1 is last array item
cell+ 2dup u>
; constant next-item

:noname ( ad -- ad ) dup @ dup * over ! ; constant squared

: init ( ad n -- ad+n' ad ) cells over + swap 1 cells - ;

: square[] ( ad n -- ) init next-item squared map 2drop ;

a[] a-len square[]

: .a[] init next-item [: ( ad -- ad ) dup @ . ;] map 2drop ;

a[] a-len .a[] .s
\ ---------------------------------

The first system to fail is GForth which gave the correct display for
example A

1 4 9 16 25 36 49 64 81 100

But failed to compile example B giving:

d:/projects/forthapps/experimental/src/for-each-while.fth:178: error:
Undefined word
{: it-xt op-xt :} for-each >>>it-xt<<< execute do[ op-xt execute ]next
Backtrace:
0 $6FFFF7FF1B8 throw
*** GForth error reported ***

i.e. it didn't recognise the local it-xt

This may be unfair to GForth as I am using version 0.7.9 20180905 as
that was the last Windows executable in the available GForth snapshots.
As I'm not set up to compile the latest GForth snapshots I'd be grateful
if someone could try running the above on a recent version.

\ --------------------------------
The other system to fail was VFX Forth version 5.43 build 4238 which, I
believe, is the latest version available for download.

VFX Forth ran example B correctly but with Example A it failed with:

Err# -22 ERR: Control structure mismatch.
Source: "src/for-each-while.fth" on line 163
-> : square[] ( ad n -- ) init next-item squared map 2drop ;
^
\ -------------------------------

The four systems that worked were SwiftForth, MinForth, Alex McDonald's
wf32 and my system. Swiftforth did need DO[ to be declared IMMEDIATE

I don't know why these 2 Forths failed, one possible reason is that they
probably carry out more sophisticated optimisations and the convoluted
control structure in the FOR-EACH definitions gets in the way. As far as
I can see the above code is all standard - perhaps I'm wrong.

--
Gerry

[Ruvim]

I see the same issue in Gforth 0.7.9_20230921

It fails to compile the following test case:

: foo {: x :} ahead begin x exit again then ;

But it compiles the cases:
: foo {: x :} ahead x exit then ;
: foo {: x :} begin x exit again ;
: foo {: x :} begin ahead x exit then again ;

>
> \ --------------------------------
> The other system to fail was VFX Forth version 5.43 build 4238 which, I
> believe, is the latest version available for download.
>
> VFX Forth ran example B correctly but with Example A it failed with:
>
> Err# -22 ERR: Control structure mismatch.
>  Source: "src/for-each-while.fth" on line 163
>  -> : square[]  ( ad n -- )  init next-item squared  map 2drop  ;
>                                                                 ^
> \ -------------------------------
>
> The four systems that worked were SwiftForth, MinForth, Alex McDonald's
> wf32 and my system. Swiftforth did need DO[ to be declared IMMEDIATE

Both examples also work in minForth/3.4.8 and SP-Forth/4.29

> I don't know why these 2 Forths failed, one possible reason is that they
> probably carry out more sophisticated optimisations and the convoluted
> control structure in the FOR-EACH definitions gets in the way. As far as
> I can see the above code is all standard - perhaps I'm wrong.
>


--

Ruvim

[Ruvim]

On 2023-12-07 00:46, Gerry Jackson wrote:

A simpler variant:

: cs-drop-dest ( CS: dest -- )

postpone true postpone until

;

>
> : for-each
>    postpone ahead postpone begin postpone 0< postpone if
>    postpone begin 3 cs-roll postpone then
> ; immediate

>
> synonym do[ while [defined] [SwiftForth] [if] immediate [then]
>
> : ]next
>    postpone repeat postpone then cs-drop-dest
> ; immediate
>
> \ The pipeline operator
> : |>  ( f -- )  3 cs-pick postpone ?dup postpone 0= postpone until  ;
> immediate

Actually, the pipeline operator has the stack diagram ( n -- )
if n = 0 -- continue the pipeline
if n < 0 -- break the pipeline, continue the loop
if n > 0 -- break both the pipeline and the loop



Explanation for other readers.

The code fragment:

for-each
...
do[
...
|>
...
]next


Is compiled as follows:

ahead begin 0< if begin [ 3 cs-roll ] then
...
while
...
[ 3 cs-pick ] ?dup 0= until
...
repeat then
ahead [ 1 cs-roll ] again then


Using the same indentation for the same control flow structure instance,
it can be written as follows:

ahead
begin
0<
if
begin
[ 3 cs-roll ]
then
...
while
...
?dup 0=
[ 3 cs-pick ]
until
...
repeat
then

ahead
[ 1 cs-roll ]
again \ this line is unreachable in run-time
then


NB: "cs-roll" and "cs-pick" allows us to intersect the different
instances of control-flow structures.



--
Ruvim

[dxf]

On 7/12/2023 10:11 am, Ruvim wrote:
> On 2023-12-07 00:46, Gerry Jackson wrote:

>> ...

>> The FOR-EACH definitions are:
>>
>> : cs-drop-dest  ( CS: dest -- )
>>     postpone ahead 1 cs-roll postpone again postpone then
>> ;
>
> A simpler variant:
>
>   : cs-drop-dest ( CS: dest -- )
>     postpone true postpone until
>   ;

IIRC cs-drop had been a 200x proposal but was ultimately shelved.
Prematurely it would appear.

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>: for-each
> postpone ahead postpone begin postpone 0< postpone if
> postpone begin 3 cs-roll postpone then
>; immediate

...

>Example B - MAP with locals for the xts
>: map ( ... iter-xt op-xt -- ... )
> {: it-xt op-xt :} for-each it-xt execute do[ op-xt execute ]next
>;

...

>d:/projects/forthapps/experimental/src/for-each-while.fth:178: error:
>Undefined word
> {: it-xt op-xt :} for-each >>>it-xt<<< execute do[ op-xt execute ]next
>Backtrace:
> 0 $6FFFF7FF1B8 throw
>*** GForth error reported ***
>
>i.e. it didn't recognise the local it-xt

This is due to a weakness in automatic scoping of locals in Gforth
(since 1994). You can read all about it in
<https://gforth.org/manual/Where-are-locals-visible-by-name_003f.html>.

You can work around it by changing FOR-EACH as follows:

[undefined] assume-live [if] : assume-live ; immediate [then]

: for-each
postpone ahead postpone assume-live postpone begin postpone 0< postpone if

postpone begin 3 cs-roll postpone then
; immediate

[Anton Ertl]

dxf <dxf...@gmail.com> writes:
>IIRC cs-drop had been a 200x proposal but was ultimately shelved.

From where do you have this information? At least the draft minutes
from the 2023 meeting say:

|Revise proposal considering the replies. Action: UH

[Gerry Jackson]

Thanks for trying it out on GForth. A later post by Anton shows that
this is a known 'problem' and gives a workaround.

>>
>> \ --------------------------------
>> The other system to fail was VFX Forth version 5.43 build 4238 which,
>> I believe, is the latest version available for download.
>>
>> VFX Forth ran example B correctly but with Example A it failed with:
>>
>> Err# -22 ERR: Control structure mismatch.
>>   Source: "src/for-each-while.fth" on line 163
>>   -> : square[]  ( ad n -- )  init next-item squared  map 2drop  ;
>>                                                                  ^
>> \ -------------------------------
>>
>> The four systems that worked were SwiftForth, MinForth, Alex
>> McDonald's wf32 and my system. Swiftforth did need DO[ to be declared
>> IMMEDIATE
>
> Both examples also work in minForth/3.4.8 and SP-Forth/4.29
>

Once again thanks

> --
> Ruvim
>

--
Gerry

[Gerry Jackson]

Yes, I never thought of that, thanks

>>
>> : for-each
>>     postpone ahead postpone begin postpone 0< postpone if
>>     postpone begin 3 cs-roll postpone then
>> ; immediate
>
>>
>> synonym do[ while [defined] [SwiftForth] [if] immediate [then]
>>
>> : ]next
>>     postpone repeat postpone then cs-drop-dest
>> ; immediate
>>
>> \ The pipeline operator
>> : |>  ( f -- )  3 cs-pick postpone ?dup postpone 0= postpone until  ;
>> immediate
>
> Actually, the pipeline operator has the stack diagram ( n -- )
>   if n = 0 -- continue the pipeline
>   if n < 0 -- break the pipeline, continue the loop
>   if n > 0 -- break both the pipeline and the loop
>

Oops, I was guilty of using the definition for => and forgot to change
the stack diagram.

I think you aren't quite correct, it should be:

|> ( n -- ) if n = 0 -- continue the pipeline

|> ( n -- n ) otherwise -- breaks the pipeline and branches back to the
first BEGIN in FOR_EACH where:
n < 0 continues the loop
n > 0 breaks the loop

Many thanks for that, I was going to do something similar today - you've
saved me the effort.

>
>
> --
> Ruvim
>

--
Gerry

[Gerry Jackson]

Thanks for pointing me to that, but is that the correct approach to take
when doing something non-standard?

What I mean is that you've implemented a nice computer science concept
in GForth which can be broken by use of a low level facility (CS-PICK
CS-ROLL AHEAD etc) provided by standard Forth. Then someone using that
facility is surprised when some standard Forth code fails.

Wouldn't a better approach be to implement the standard correctly but
offer the improved locals visibility approach as an option.

Having said that I implemented a simpler visibility approach by only
allowing access to a local in a control block e.g.

: x if {: a :} a . else a 2* . then ; \ Not allowed in ANS Forth.
fails to compile with:

: x if {: a :} a . else a 2* . then ;
^ Undefined word

Equally non-standard but I have the excuse of being the only user of my
system and I rarely use locals anyway.

--
Gerry

[minforth]

Why non-standard?

13.3.3.2 Syntax restrictions
b) The position in program source at which the sequence of
(LOCAL) messages is sent, referred to here as the point at
which locals are declared, shall not lie within the scope
of any control structure;

[dxf]

On 7/12/2023 6:06 pm, Anton Ertl wrote:
> dxf <dxf...@gmail.com> writes:
>> IIRC cs-drop had been a 200x proposal but was ultimately shelved.
>
> From where do you have this information?

According to the website it was first proposed for standardization
in 2017, last revised 2019, and last commented 2020.

> At least the draft minutes
> from the 2023 meeting say:
>
> |Revise proposal considering the replies. Action: UH

Was it actioned? Or perhaps that's for the 2024 meeting.

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>On 07/12/2023 06:54, Anton Ertl wrote:
>> This is due to a weakness in automatic scoping of locals in Gforth
>> (since 1994). You can read all about it in
>> <https://gforth.org/manual/Where-are-locals-visible-by-name_003f.html>.

...

>Thanks for pointing me to that, but is that the correct approach to take
>when doing something non-standard?

When a program does something non-standard, it's up to the system how
to react, but in this case this aspect of the program is standard, and
Gforth does not work as required by the standard.

>What I mean is that you've implemented a nice computer science concept
>in GForth which can be broken by use of a low level facility (CS-PICK
>CS-ROLL AHEAD etc) provided by standard Forth. Then someone using that
>facility is surprised when some standard Forth code fails.
>
>Wouldn't a better approach be to implement the standard correctly but
>offer the improved locals visibility approach as an option.

I think the right solution is to assume the following visibility on
AHEAD BEGIN: the locals visible at the most recent place where nothing
was on the control-flow stack. So the locals defined according to the
standard are visible in the whole colon definition after the
definition, as the standard requires.

This should not make anything visible that must not be visible
according to the principles behind automatic scoping: If a local is
visible at a point without control-flow (or SCOPE), everything
afterwards in the definition is only reachable through that point (or
it is unreachable, in which case the visibility is irrelevant), so the
locals visible at that point are also visible there.

This can actually be relaxed a little more: use locals visible at the
last place P where only dests are on the control-flow stack (and the
LEAVE stack is empty). Dests correspond to backwards edges in the
control-flow graph, so everything afterwards is only reachable through
P.

[Gerry Jackson]

My second attempt at a reply. The earlier reply got lost when
Thunderbird seemed to stop responding. Meanwhile Anton's latest reply
made some of my reply redundant.

What I was trying to say was that Gforth requiring a workaround to
compile a standard section of code is not user friendly irrespective of
whether Gforth is non-standard.

As Gforth can declare locals within control structures it is clearly
non-standard as the restriction you quote uses 'shall not' instead of
making it an ambiguous condition. I don't see why that restriction is
there (apart from within DO loops which traditionally use the return
stack) as declaring locals like that can be useful, for example in WHILE
loops with the pattern:

(ad1 ad2) begin {: a b :} ... while ... a char+ b char+ repeat ...

where the updated locals get auto loaded without needing TO

--
Gerry

[Gerry Jackson]

It appears that you have a solution to the problem. However despite me
having a way of scoping locals and not remembering why or ever using it,
I'm curious about why you think auto-scoping locals is worth the trouble
when the philosophy of Forth in general is 'let the programmer
beware'. Apart from it being an interesting problem to solve of course.

--
Gerry

[minforth]

Gerry Jackson wrote:

> As Gforth can declare locals within control structures it is clearly
> non-standard as the restriction you quote uses 'shall not' instead of
> making it an ambiguous condition. I don't see why that restriction is
> there (apart from within DO loops which traditionally use the return
> stack) as declaring locals like that can be useful, for example in WHILE
> loops with the pattern:

> (ad1 ad2) begin {: a b :} ... while ... a char+ b char+ repeat ...

> where the updated locals get auto loaded without needing TO

The standard distinguishes between <arg> and <val> locals. The latter
are not initialised; in the locals declaration they appear after a
vertical bar character.

IIRC in gforth you can use {: ... :} multiple times within a word
definition. It would therefore be perfectly possible to use {: | b :}
within a control structure without logical conflicts.

Personally, I find the whole thing rather less helpful.

[dxf]

Leaders are exempt from following. Otherwise there'd be no creativity.

[Gerry Jackson]

I'm not sure what you mean by 'the whole thing'. Is it the locals in
general, Gforth extensions to locals or the FOR-EACH control structure?

If the latter, that's ok. I posted it in the hope of getting feedback
whether positive or negative.

If you mean locals I agree with you. As I've stated elsewhere I
incorporated extensions to locals into my system many years ago and have
found that I hardly ever use locals, let alone the extensions. I only
introduced them into this discussion as I thought the initial definition
of MAP where 2 xts were put on the R stack looked a bit awkward and so
tried it with locals only to discover to my astonishment that two well
respected Forth systems didn't work. Hence the diversion.

Two things about FOR-EACH I don't think I've yet mentioned.

The pipeline operator is optional. Without it the whole thing reverts to
a BEGIN ... WHILE ... REPEAT loop with a small amount of redundant code.
So why use FOR-EACH, IMHO the FOR-EACH is arguably more readable.

The pipeline operator can make the FOR-EACH statement equivalent to
multi-WHILE loops without the drawbacks of ELSE parts being separated in
the source code. In addition using IF statements before the |> operator
the programmer has the option of continuing the loop, breaking the
pipeline and continuing or breaking the loop. This is also true of
Gforth's CASE extensions but I would argue readability again.

--
Gerry

[none albert]

In article <eca9d334109ee6b7...@news.novabbs.com>,
minforth <minf...@gmx.net> wrote:
>Gerry Jackson wrote:
<SNIP>

>
>IIRC in gforth you can use {: ... :} multiple times within a word
>definition. It would therefore be perfectly possible to use {: | b :}
>within a control structure without logical conflicts.

No. I'm conflicted about defining locals once, or defining
locals as governed by the control structure. In a loop
that results in multiple instances. It is sufficiently weird
to get away from.

Groetjes Albert
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat spinning. - the Wise from Antrim -

[Ruvim]

On 2023-12-08 01:47, Gerry Jackson wrote:
>
>
> As Gforth can declare locals within control structures it is clearly
> non-standard as the restriction you quote uses 'shall not' instead of
> making it an ambiguous condition.

In both cases (i.e., regardless of the form), it is non-standard for
*programs* and only for programs. A system is allowed to provide this
capability, and non-standard programs can use this capability.

Essentially, "shall not" means that the behavior in case of violation of
this restriction depends on the Forth system.

The only difference from a formal ambiguous condition is that the
behavior in the case of an ambiguous condition shall be documented.


The section 4 Documentation requirements [1] says:
"When it is impossible or infeasible for a system or program to
define a particular behavior itself, it is permissible to state that the
behavior is unspecifiable and to explain the circumstances and reasons
why this is so."

The section 4.1.2 Ambiguous conditions [2] says:
"A system shall document the system action taken upon each of the
general or specific ambiguous conditions identified in this standard."


BTW, I created a poll on ForthHub [3] to find out if it is important for
users to have this documentation. But no feedback has been received yet.


[1] https://forth-standard.org/standard/doc#chapter.4
[2] https://forth-standard.org/standard/doc#subsection.4.1.2
[3] https://github.com/ForthHub/discussion/discussions/154

[Ruvim]

On 2023-12-08 12:11, Gerry Jackson wrote:
[...]

>
> Two things about FOR-EACH I don't think I've yet mentioned.
>
> The pipeline operator is optional. Without it the whole thing reverts to
> a BEGIN ... WHILE ... REPEAT loop with a small amount of redundant code.
> So why use FOR-EACH, IMHO the FOR-EACH is arguably more readable.
>
> The pipeline operator can make the FOR-EACH statement equivalent to
> multi-WHILE loops without the drawbacks of ELSE parts being separated in
> the source code.

OTOH, FOR-EACH part is also optional. A pipeline operator can be used
instead:

FOR-EACH TRUE DO[ next-item |> check-item |> process-item ]NEXT

It's not obvious to me that placing "next-item" into a separate section
rather than into the chain (like in this example) makes readability better.

Also, I would use different operators to break and to continue the loop,
instead of different signs of the input value.

> In addition using IF statements before the |> operator
> the programmer has the option of continuing the loop, breaking the
> pipeline and continuing or breaking the loop. This is also true of
> Gforth's CASE extensions but I would argue readability again.
>

A small clarification: the operator "|>" cannot be placed inside an "IF"
statement. Only its input value can be calculated (using "IF", or any
other words).


--
Ruvim

[Ruvim]

On 2023-12-08 14:25, Ruvim wrote:
> On 2023-12-08 01:47, Gerry Jackson wrote:
>>
>>
>> As Gforth can declare locals within control structures it is clearly
>> non-standard as the restriction you quote uses 'shall not' instead of
>> making it an ambiguous condition.
>
> In both cases (i.e., regardless of the form), it is non-standard for
> *programs* and only for programs. A system is allowed to provide this
> capability, and non-standard programs can use this capability.

Well, probably I was not quite correct here.

The section "3 Usage requirements"
<https://forth-standard.org/standard/usage#usage> says:
"A program that requires a system to provide words or techniques not
defined in this standard has an environmental dependency."

So, if a program requires a specific behavior in the case of some
ambiguous condition, it can be considered as just an environmental
dependency. Especially, if this requirement is mentioned in the
program's documentation.

>
> Essentially, "shall not" means that the behavior in case of violation of
> this restriction depends on the Forth system.
>
> The only difference from a formal ambiguous condition is that the
> behavior in the case of an ambiguous condition shall be documented.
>
>
> The section 4 Documentation requirements [1] says:
>   "When it is impossible or infeasible for a system or program to
> define a particular behavior itself, it is permissible to state that the
> behavior is unspecifiable and to explain the circumstances and reasons
> why this is so."
>
> The section 4.1.2 Ambiguous conditions [2] says:
>   "A system shall document the system action taken upon each of the
> general or specific ambiguous conditions identified in this standard."
>
>
> BTW, I created a poll on ForthHub [3] to find out if it is important for
> users to have this documentation. But no feedback has been received yet.
>
>
> [1] https://forth-standard.org/standard/doc#chapter.4
> [2] https://forth-standard.org/standard/doc#subsection.4.1.2
> [3] https://github.com/ForthHub/discussion/discussions/154
>

--
Ruvim

[minforth]

Gerry Jackson wrote:
> On 08/12/2023 01:22, minforth wrote:
>> The standard distinguishes between <arg> and <val> locals. The latter
>> are not initialised; in the locals declaration they appear after a
>> vertical bar character.
>>
>> IIRC in gforth you can use {: ... :} multiple times within a word
>> definition. It would therefore be perfectly possible to use {: | b :}
>> within a control structure without logical conflicts.
>>
>> Personally, I find the whole thing rather less helpful.

> I'm not sure what you mean by 'the whole thing'. Is it the locals in
> general, Gforth extensions to locals or the FOR-EACH control structure?

Sorry if I was unclear. I meant gforth's extensions. The FOR-EACH
structure you suggest can be quite practical.

Since I often have many items on the stack that can't be factored into
smaller parts, my code would be illegible without locals. But I have
added structure and array locals for this.

IMO a locals declaration is a "talking stack comment" right after the
header of a word. This partial documentation would fall apart if I
declared more and more locals in other parts of the code, let alone
inside control structures.

So, at least for me, gforth's locals extension dicussed here wouldn't be
helpful. For others it may work fine of course.

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>What I was trying to say was that Gforth requiring a workaround to
>compile a standard section of code is not user friendly irrespective of
>whether Gforth is non-standard.

Gforth is currently non-standard, because it does not compile the
standard program

: foo {: x :} ahead begin x again then ;

>As Gforth can declare locals within control structures it is clearly
>non-standard as the restriction you quote uses 'shall not' instead of
>making it an ambiguous condition.

As Ruvim writes, this is a restriction on programs, and standard
systems are allowed to accept non-standard programs.

Whether the document uses "shall not" or "ambiguous condition" makes
no difference in that respect. The chapter on "The optional Locals
word set" is written in a different style from the rest of the
standard, and this usage is just one of the places where this shows
up.

>I don't see why that restriction is
>there

The long list of restrictions is there to make it easier for system
implementors to implement the wordset. In this case: locals will be
visible until the end of the definition, which makes implementing
visibility easier. And locals also live until the end of the
definition, which makes it easier to implement how they are
deallocated.

>(apart from within DO loops which traditionally use the return
>stack)

And yes, it also means that locals can be implemented on the return
stack without I and J having to consider their existence.

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>I'm curious about why you think auto-scoping locals is worth the trouble
> when the philosophy of Forth in general is 'let the programmer
>beware'.

I don't care that much for claims about "the philosophy of Forth". I
have seen too much nonsense advocated with this claimed philosophy;
things along the lines of: The philosophy demands that we do it this
way, because it makes this particular word shorter by a few bytes; who
cares if this word is used dozens of times, and every use becomes
harder, more error-prone, or even bigger or slower. I care!

Anyway, you have given an example of why it is useful to define locals
inside control structures in <ukteh9$1dfna$1...@dont-email.me>, and I use
that a lot. Now, if we want that, but are also required by the
standard to implement general control flow as discussed in A.3.2.3.2,
I need to define the lifetime and visibility of the locals, and
ideally they should live at least as long as they are visible. I
found a solution to this problem [ertl94l], so I implemented it.

Now, 29 years later, there is the first complaint about the main
weakness of the solution, but fortunately it is possible to fix that,
too.

@InProceedings{ertl94l,
author = "M. Anton Ertl",
title = "Automatic Scoping of Local Variables",
booktitle = "EuroForth~'94 Conference Proceedings",
year = "1994",
address = "Winchester, UK",
pages = "31--37",
url = "http://www.complang.tuwien.ac.at/papers/ertl94l.ps.gz",
abstract = "In the process of lifting the restrictions on using
locals in Forth, an interesting problem poses
itself: What does it mean if a local is defined in a
control structure? Where is the local visible? Since
the user can create every possible control structure
in ANS Forth, the answer is not as simple as it may
seem. Ideally, the local is visible at a place if
the control flow {\em must} pass through the
definition of the local to reach this place. This
paper discusses locals in general, the visibility
problem, its solution, the consequences and the
implementation as well as related programming style
questions."

[Anton Ertl]

minf...@gmx.net (minforth) writes:
>The standard distinguishes between <arg> and <val> locals. The latter
>are not initialised; in the locals declaration they appear after a
>vertical bar character.
>
>IIRC in gforth you can use {: ... :} multiple times within a word
>definition. It would therefore be perfectly possible to use {: | b :}
>within a control structure without logical conflicts.

Yes, it's possible. What's your point?

The "| ..." part of the {:...:} syntax is unnecessary. Instead of

{: ... | x y z :}

one could write

0 0 0 {: ... x y z :}

But I don't need either in my code, because, by being able to define
locals anywhere, I define them when the initial value is known, so no
need to define it earlier uninitialized and then set the value later.

Looking in the Gforth sources, the use of '|' in a locals definition
is usually for cases where a local buffer is defined with the 'name['
syntax.

[Gerry Jackson]

On 08/12/2023 10:47, Ruvim wrote:
> On 2023-12-08 12:11, Gerry Jackson wrote:
> [...]
>>
>> Two things about FOR-EACH I don't think I've yet mentioned.
>>
>> The pipeline operator is optional. Without it the whole thing reverts
>> to a BEGIN ... WHILE ... REPEAT loop with a small amount of redundant
>> code. So why use FOR-EACH, IMHO the FOR-EACH is arguably more readable.
>>
>> The pipeline operator can make the FOR-EACH statement equivalent to
>> multi-WHILE loops without the drawbacks of ELSE parts being separated
>> in the source code.
>
> OTOH, FOR-EACH part is also optional. A pipeline operator can be used
> instead:
>
>   FOR-EACH TRUE DO[ next-item |> check-item |> process-item ]NEXT
>
> It's not obvious to me that placing "next-item" into a separate section
> rather than into the chain (like in this example) makes readability better.

You're probably right, my intention was to highlight that the first item
had to be something that iterated through a collection. But without
thinking it through I guess that's a fairly minor change to make. It
would also eliminate the need for DO[

>
> Also, I would use different operators to break and to continue the loop,
> instead of different signs of the input value.
>

Do you mean three operators that could be placed between the pipeline
operationsi.e. retaining the
iterator pipe-operator operation pipe-operator ... sequence

where we have three different pipe operators that, for the sake of
argument, might be:

|> for Proceed currently handled by a value 0
<|x for exit pipeline back, currently <0, like C continue
|x> for exit pipeline forward, currently >0, like C break

But without thinking about it deeply I don't see how that would work as
the decision about which option to take is at run-time not compile-time.

Whether this is managed by conditional jumps or possibly quotations ISTM
that there is always a choice of 1 from 3 has to be made at runtime.

Or did you have another idea in mind?

>
>> In addition using IF statements before the |> operator the programmer
>> has the option of continuing the loop, breaking the pipeline and
>> continuing or breaking the loop. This is also true of Gforth's CASE
>> extensions but I would argue readability again.
>>
>
> A small clarification: the operator "|>" cannot be placed inside an "IF"
> statement. Only its input value can be calculated (using "IF", or any
> other words).
>

Yes that is correct in general. If the IF was in the pipeline code it
could possibly be accomodated using a CS-ROLL or CS-PICK but of course
it could be in a separate colon definition when it would be impossible.
I was envisioning the arms of the IF ... ELSE ... THEN generating the
appropriate one of the three possibilites <0 0 >0 to be handled by the
following |>.

>
> --
> Ruvim
>

Sorry but I won't be able to work on this again until Sunday. Thanks for
your comments, it's certainly making me think about it more so we should
hopefully end up with a better solution.

--
Gerry

[Gerry Jackson]

On 08/12/2023 14:44, Anton Ertl wrote:
> Gerry Jackson <do-no...@swldwa.uk> writes:
>> I'm curious about why you think auto-scoping locals is worth the trouble
>> when the philosophy of Forth in general is 'let the programmer
>> beware'.
>
> I don't care that much for claims about "the philosophy of Forth". I
> have seen too much nonsense advocated with this claimed philosophy;
> things along the lines of: The philosophy demands that we do it this
> way, because it makes this particular word shorter by a few bytes; who
> cares if this word is used dozens of times, and every use becomes
> harder, more error-prone, or even bigger or slower. I care!
>
> Anyway, you have given an example of why it is useful to define locals
> inside control structures in <ukteh9$1dfna$1...@dont-email.me>, and I use
> that a lot. Now, if we want that, but are also required by the
> standard to implement general control flow as discussed in A.3.2.3.2,
> I need to define the lifetime and visibility of the locals, and
> ideally they should live at least as long as they are visible. I
> found a solution to this problem [ertl94l], so I implemented it.
>
> Now, 29 years later, there is the first complaint about the main
> weakness of the solution, but fortunately it is possible to fix that,
> too.

I seem to remember years ago that I caused you to improve code generated
by Gray many years after Gray had been developed. I'm getting to be a
nuisance :)

>
> @InProceedings{ertl94l,
> author = "M. Anton Ertl",
> title = "Automatic Scoping of Local Variables",
> booktitle = "EuroForth~'94 Conference Proceedings",
> year = "1994",
> address = "Winchester, UK",
> pages = "31--37",
> url = "http://www.complang.tuwien.ac.at/papers/ertl94l.ps.gz",
> abstract = "In the process of lifting the restrictions on using
> locals in Forth, an interesting problem poses
> itself: What does it mean if a local is defined in a
> control structure? Where is the local visible? Since
> the user can create every possible control structure
> in ANS Forth, the answer is not as simple as it may
> seem. Ideally, the local is visible at a place if
> the control flow {\em must} pass through the
> definition of the local to reach this place. This
> paper discusses locals in general, the visibility
> problem, its solution, the consequences and the
> implementation as well as related programming style
> questions."
> }
>

I see, a research topic

--
Gerry

[dxf]

On 8/12/2023 10:37 pm, Ruvim wrote:
> On 2023-12-08 14:25, Ruvim wrote:
>> On 2023-12-08 01:47, Gerry Jackson wrote:
>>>
>>>
>>> As Gforth can declare locals within control structures it is clearly non-standard as the restriction you quote uses 'shall not' instead of making it an ambiguous condition.
>>
>> In both cases (i.e., regardless of the form), it is non-standard for *programs* and only for programs. A system is allowed to provide this capability, and non-standard programs can use this capability.
>
> Well, probably I was not quite correct here.
>
> The section "3 Usage requirements" <https://forth-standard.org/standard/usage#usage> says:
>   "A program that requires a system to provide words or techniques not defined in this standard has an environmental dependency."
>
> So, if a program requires a specific behavior in the case of some ambiguous condition, it can be considered as just an environmental dependency. Especially, if this requirement is mentioned in the program's documentation.

A program that relies on an 'ambiguous condition' can hardly be portable.
If ANS wants to claim documenting it will make it 'standard' that's their
business and yours if you choose to buy into it.

[Anton Ertl]

dxf <dxf...@gmail.com> writes:
>A program that relies on an 'ambiguous condition' can hardly be portable.

That depends on the ambiguous condition. In particular, looking at
<https://forth-standard.org/standard/doc#subsection.4.1.2>, I see

|producing a result out of range, e.g., multiplication (using *)
|results in a value too big to be represented by a single-cell integer
|(6.1.0090 *, 6.1.0100 */, 6.1.0110 */MOD, 6.1.0570 >NUMBER, 6.1.1561
|FM/MOD, 6.1.2214 SM/REM, 6.1.2370 UM/MOD, 8.6.1.1820 M*/);

Certainly for * every system with a given cell size produces the same
result. This has led to the result of * overflow being standardized
in the next release <http://www.forth200x.org/twos-complement.html>.
It's interesting that this ambiguous condition does not mention +, -,
or it's variants.

Bottom line: In some cases there is common practice even if it has not
been standardized.

I don't think that the documentation in 4.1.2 is useful to find such
common practice, though.

>If ANS wants to claim documenting it will make it 'standard' that's their
>business and yours if you choose to buy into it.

I don't know who this ANS is and why he/she would "want to claim" such
a thing, and why anybody should care about such a supposed urge, if
he/she does not actually make such a claim.

In any case, Forth-94 and Forth-2012 require standard systems to
document some ambiguous conditions. As you point out, that certainly
does not help in writing portable programs.

If the behaviour is considered a feature by the Forth system
implementor, the behaviour will certainly be documented without such a
requirement. If it is not a feature, what use should a programmer
have from the documentation?

[dxf]

On 9/12/2023 6:18 pm, Anton Ertl wrote:
> dxf <dxf...@gmail.com> writes:
>> A program that relies on an 'ambiguous condition' can hardly be portable.
>
> That depends on the ambiguous condition. In particular, looking at
> <https://forth-standard.org/standard/doc#subsection.4.1.2>, I see
>
> |producing a result out of range, e.g., multiplication (using *)
> |results in a value too big to be represented by a single-cell integer
> |(6.1.0090 *, 6.1.0100 */, 6.1.0110 */MOD, 6.1.0570 >NUMBER, 6.1.1561
> |FM/MOD, 6.1.2214 SM/REM, 6.1.2370 UM/MOD, 8.6.1.1820 M*/);
>
> Certainly for * every system with a given cell size produces the same
> result. This has led to the result of * overflow being standardized
> in the next release <http://www.forth200x.org/twos-complement.html>.
> It's interesting that this ambiguous condition does not mention +, -,
> or it's variants.
>
> Bottom line: In some cases there is common practice even if it has not
> been standardized.
>
> I don't think that the documentation in 4.1.2 is useful to find such
> common practice, though.

It was 'ambiguous' because the result was mathematically incorrect.
You can prescribe the actions taken but it won't change the fact.

> ...

> In any case, Forth-94 and Forth-2012 require standard systems to
> document some ambiguous conditions. As you point out, that certainly
> does not help in writing portable programs.

Unfortunately ANS (aka ANS-FORTH) was more interested including as many
systems under their umbrella than they were in users writing portable
programs.

[Anton Ertl]

dxf <dxf...@gmail.com> writes:
>On 9/12/2023 6:18 pm, Anton Ertl wrote:
>> That depends on the ambiguous condition. In particular, looking at
>> <https://forth-standard.org/standard/doc#subsection.4.1.2>, I see
>>
>> |producing a result out of range, e.g., multiplication (using *)
>> |results in a value too big to be represented by a single-cell integer
>> |(6.1.0090 *, 6.1.0100 */, 6.1.0110 */MOD, 6.1.0570 >NUMBER, 6.1.1561
>> |FM/MOD, 6.1.2214 SM/REM, 6.1.2370 UM/MOD, 8.6.1.1820 M*/);
>>
>> Certainly for * every system with a given cell size produces the same
>> result. This has led to the result of * overflow being standardized
>> in the next release <http://www.forth200x.org/twos-complement.html>.
>> It's interesting that this ambiguous condition does not mention +, -,
>> or it's variants.
>>
>> Bottom line: In some cases there is common practice even if it has not
>> been standardized.
>>
>> I don't think that the documentation in 4.1.2 is useful to find such
>> common practice, though.
>
>It was 'ambiguous' because the result was mathematically incorrect.

Where do I find the documentation on the reasons for this ambiguous
condition that your claim is based on?

I don't have such documentation, so I can only guess. And my guess is
that it's because Forth-94 allowed systems to choose among three
different representations of negative numbers: 2's-complement,
1s-complement, sign-magnitude. And these representations result in
different behaviour on overflow, so the easiest way was to declare
this to be an ambiguous condition.

In any case, it does not matter whether you consider the result to be
"mathematically incorrect", modulo arithmetic produces results that
are useful for some applications (e.g., for hash functions), so given
that it is common practice, it is a good idea to standardize it.

[dxf]

On 9/12/2023 10:24 pm, Anton Ertl wrote:
> dxf <dxf...@gmail.com> writes:
>> On 9/12/2023 6:18 pm, Anton Ertl wrote:
>>> That depends on the ambiguous condition. In particular, looking at
>>> <https://forth-standard.org/standard/doc#subsection.4.1.2>, I see
>>>
>>> |producing a result out of range, e.g., multiplication (using *)
>>> |results in a value too big to be represented by a single-cell integer
>>> |(6.1.0090 *, 6.1.0100 */, 6.1.0110 */MOD, 6.1.0570 >NUMBER, 6.1.1561
>>> |FM/MOD, 6.1.2214 SM/REM, 6.1.2370 UM/MOD, 8.6.1.1820 M*/);
>>>
>>> Certainly for * every system with a given cell size produces the same
>>> result. This has led to the result of * overflow being standardized
>>> in the next release <http://www.forth200x.org/twos-complement.html>.
>>> It's interesting that this ambiguous condition does not mention +, -,
>>> or it's variants.
>>>
>>> Bottom line: In some cases there is common practice even if it has not
>>> been standardized.
>>>
>>> I don't think that the documentation in 4.1.2 is useful to find such
>>> common practice, though.
>>
>> It was 'ambiguous' because the result was mathematically incorrect.
>
> Where do I find the documentation on the reasons for this ambiguous
> condition that your claim is based on?

From the link you provided:

4.1.2 Ambiguous conditions
...

producing a result out of range, e.g., multiplication (using *) results
in a value too big to be represented by a single-cell integer

>

> I don't have such documentation, so I can only guess. And my guess is
> that it's because Forth-94 allowed systems to choose among three
> different representations of negative numbers: 2's-complement,
> 1s-complement, sign-magnitude. And these representations result in
> different behaviour on overflow, so the easiest way was to declare
> this to be an ambiguous condition.
>
> In any case, it does not matter whether you consider the result to be
> "mathematically incorrect", modulo arithmetic produces results that
> are useful for some applications (e.g., for hash functions), so given
> that it is common practice, it is a good idea to standardize it.

And the result will still be 'out of range' which qualifies it as an
'ambiguous condition' under ANS' definition of the term.

[Ruvim]

On 2023-12-09 00:49, Gerry Jackson wrote:
> On 08/12/2023 10:47, Ruvim wrote:
>> On 2023-12-08 12:11, Gerry Jackson wrote:
>> [...]
>>>
>>> Two things about FOR-EACH I don't think I've yet mentioned.
>>>
>>> The pipeline operator is optional. Without it the whole thing reverts
>>> to a BEGIN ... WHILE ... REPEAT loop with a small amount of redundant
>>> code. So why use FOR-EACH, IMHO the FOR-EACH is arguably more readable.
>>>
>>> The pipeline operator can make the FOR-EACH statement equivalent to
>>> multi-WHILE loops without the drawbacks of ELSE parts being separated
>>> in the source code.
>>
>> OTOH, FOR-EACH part is also optional. A pipeline operator can be used
>> instead:
>>
>>    FOR-EACH TRUE DO[ next-item |> check-item |> process-item ]NEXT
>>
>> It's not obvious to me that placing "next-item" into a separate
>> section rather than into the chain (like in this example) makes
>> readability better.
>
> You're probably right, my intention was to highlight that the first item
> had to be something that iterated through a collection. But without
> thinking it through I guess that's a fairly minor change to make. It
> would also eliminate the need for DO[
>
>>
>> Also, I would use different operators to break and to continue the
>> loop, instead of different signs of the input value.
>>
>
> Do you mean three operators that could be placed between the pipeline

> operations i.e. retaining the

>    iterator pipe-operator operation pipe-operator ... sequence
>
> where we have three different pipe operators that, for the sake of
> argument, might be:
>
>    |>  for Proceed currently handled by a value 0
>   <|x  for exit pipeline back, currently <0, like C continue
>    |x> for exit pipeline forward, currently >0, like C break

>
> But without thinking about it deeply I don't see how that would work as
> the decision about which option to take is at run-time not compile-time.
>
> Whether this is managed by conditional jumps or possibly quotations ISTM
> that there is always a choice of 1 from 3 has to be made at runtime.
>
> Or did you have another idea in mind?
>

I mean a compile-time decision. My arguments are as follows.

1. In most use cases, you know at compile-time whether you need to
continue or break the loop (I mean, in the case of a pipeline break).
Then, it's better for readability to use different pipe-operators for
these cases.

2. For code reuse, it's better if an operation returns (and a
pipe-operator accepts) a value of one of two disjoint types {0, x/0},
rather than one of three disjoint types {0, +n/0, -n/0} (where 0 is a
singleton {"0"}, and "\" means a set difference).


If a pipe-operator distinguishes two types of the input value, two
operators are enough. And if we use zero to break a pipeline, these
operators can be defined via your "|>" as follows:

: |>| ( 0|x -- ) postpone 0= postpone |> ; immediate
: |<| ( 0|x -- ) postpone 0= postpone abs postpone |> ; immediate



--
Ruvim

[none albert]

In article <ul5967$2qirl$1...@dont-email.me>,

Components of a pipe line must not contain logic to decide when to
stop. It should terminate as the input terminates.
So pipeline operators should marry a yield operation that produces a
stream of data, and decides termination.

>
>
>
>--
>Ruvim

[Gerry Jackson]

Why do you say that's better for code reuse?

>
>
> If a pipe-operator distinguishes two types of the input value, two
> operators are enough. And if we use zero to break a pipeline, these
> operators can be defined via your "|>" as follows:
>
>   : |>|  ( 0|x -- ) postpone 0= postpone |> ; immediate
>   : |<|  ( 0|x -- ) postpone 0= postpone abs postpone |> ; immediate
>

Thanks for that. I'm not convinced that your suggestion is better for
readability. Defining suitable constants for pipeline break and continue
might suffice.

However I will experiment with your suggestions, including removing DO[
so that the iterator is the start of the pipeline, when I get a bit more
time. I'm committed to doing some work on a website I help to maintain
for the next few weeks (programming in PHP) so it won't be until next year.

--
Gerry

[Hans Bezemer]

On 13-12-2023 22:47, Gerry Jackson wrote:
OK, I made a high level Forth version WITH size.
You only get the address, that's it.

It's full of 4tH-isms, but I guess one will manage.

[UNDEFINED] foreach [IF]
\ 'f ( addr --)
: foreach ( 'f addr count size -- )
tuck * >r -rot r> bounds ?do
i over execute over ( size 'f size)
+loop drop drop ( --)
;

\ 'f ( st addr -- st')
: reduce ( st 'f addr count size -- st')
tuck * >r -rot >r rot r> r> bounds ?do
over i swap execute >r over r> swap
+loop nip nip ( st')
;

aka foreach map
[THEN]

struct
field: one
field: two
end-struct /bla

6 constant #bla

#bla /bla * array bla
[: dup bla - swap -> two ! ;] bla #bla /bla foreach
[: -> two ? ;] bla #bla /bla foreach depth . cr
[: -> two dup @ dup * swap ! ;] bla #bla /bla map depth . cr
[: -> two ? ;] bla #bla /bla foreach depth . cr
0 [: -> two @ + ;] bla #bla /bla reduce . depth .

Hans Bezemer

[none albert]

In article <nnd$46aa01b1$22ce9a16@98c43956ef3e3602>,

If you show a test, show the result of the test.

>
>Hans Bezemer

Groetjes Albert