CfV: Quotations

[Anton Ertl]

This is actually a poll about how widely the proposal is implemented
and how popular it is among the programmers. It is called a CfV
(call-for-votes) because the process is inspired by the Usenet Rdf/CfV
process.

You find the actual ballot further down (look for "VOTING INSTRUCTIONS"),
after the proposal on which you vote.

Votetaker: Anton Ertl <an...@mips.complang.tuwien.ac.at>

Change history
--------------
28Jul2015 Alex McDonald v1 initial draft
04Aug2015 Alex McDonald v2 update
added & extended "Rationale", "Discussion"
formalised "Specification"
corrected "Implementation"
added "References"
01Jul2016 Anton Ertl v3
various non-substantive changes throughout
removed revised definition of RECURSE
revised the stack effect to have quotation-sys colon-sys
revised the specification wording
added tests
28Jul2016 Anton Ertl v4
";" and ";CODE" in quotations are now ambiguous (v3 ToDo result)
Added q6 test (please try it!)
minor changes
10Aug2016 Anton Ertl CfV
2 new tests, no other changes

Quotations
==========

The essence of quotations is to provide nested colon definitions,
in which the inner definition(s) are nameless. The expression

: foo ... [: some words ;] ... ;

is equivalent to

:noname some words ; Constant #temp#
: foo ... #temp# ... ;

A simple quotation is an anonymous colon definition that is defined
inside a colon definition or quotation.

Rationale
---------

There is no situation where quotations cannot be implemented by
existing Forth words. Specifcially, :NONAME and : provide all the
features proposed for quotations. Their advantage is as a syntactic
"sugar" that permits a nameless definition in close proximity to its
use; and that it avoids generating one-use names only for the purpose
of refering to the definition inside another word.

One example use of quotations is to provide a solution to the use
of CATCH in a form close to other languages' TRY ... EXCEPT
blocks.

: hex. ( u -- )
base @ >r
[: hex u. ;] catch
r> base ! throw ;

The advantage of using CATCH here is that the BASE is restored even if
there is an exception (e.g., a user interrupt) during the "U.".

Moreover, return-address manipulation has often been used as a way to
split a definition into several parts, e.g,:

: foo bar list> bla blub ;

where LIST> is a return-address manipulating word and executes the BLA
BLUB part of the word possibly several times. This demonstrates that
introducing a helper definition is unattractive to these programmers;
with quotations this code could be written without helper word as

: foo bar [: bla blub ;] map-list ;

The advantages of this variant are: 1) Implementating quotations puts
less restrictions on the Forth system than implementing manipulable
return-addresses (which would restrict tail-call elimination and
inlining). 2) It is immediately visible to the reader that there is a
separate definition containing BLA BLUB.

Experience
----------

Under the term "Phrases", quotations have been in MacForth since at
least 1987. They are documented in the infamous "missing chapter" of
the MacForth manual.

Implementations exist in (at least) VFX, Gforth and iForth, and in the
iForth case appear to have some significant use and history. The
Gforth development version from 2016-06-28 contains 70 occurences of
";]".

Interactions
------------

Quotations change the "current definition" that a number of words (in
particular, RECURSE, DOES> and locals) refer to, so we have to look at
the interactions of these words and locals.

In these interactions there is a conflict between interface simplicity
and implementation simplicity. Disallowing all combinations of
quotations and these words would give most freedom to implementors to
choose the simplest implementation, but it would make for an interface
full of blind alleys: If you choose one feature, you cannot choose any
of the others; and if you need another feature, you have to back out
of using the first one. Moreover, all these restrictions would have
to be specified in the standard.

This is not desirable, so here we propose to make all these words work
for quotations as well as they work for standalone definitions; this
is relatively cheap to implement.


RECURSE

RECURSE inside a quotation calls the quotation directly surrounding
the RECURSE, not the containing colon definition.

DOES>

A DOES> part inside a quotation ends with the terminating ";]"

Using DOES> is actually a useful example of using quotations. E.g.,
<https://www.complang.tuwien.ac.at/forth/struct.fs> contains two
helper words (DOFIELD and DOZEROFIELD) in the following code:

: dofield ( -- )
does> ( name execution: addr1 -- addr2 )
@ + ;

: dozerofield ( -- )
immediate
does> ( name execution: -- )
drop ;

: field ( align1 offset1 align size "name" -- align2 offset2 )
\ name execution: addr1 -- addr2
2 pick >r \ this uglyness is just for optimizing with dozerofield
create-field
r> if \ offset<>0
dofield
else
dozerofield
then ;

Quotations allow to write this more compactly as:

: field ( align1 offset1 align size "name" -- align2 offset2 )
\ name execution: addr1 -- addr2
2 pick >r \ this uglyness is just for optimizing with dozerofield
create-field
r> if \ offset<>0
[: does> ( addr1 -- addr2 ) @ + ;]
else
immediate [: does> ( -- ) drop ;]
then
execute ;

The second RfD proposed to allow RECURSE after DOES> in general
(currently it is ambiguous), but this is not common practice yet, so
the present version removed this part of the proposal in order to
focus on the quotations proper.

Locals

A quotation may not be able to access the locals of the outer word
because it has no knowledge of when it might be executed and hence
whether outer locals are still alive. It does permit defining and
accessing its own locals. Note that this means that both the
quotation and the containing definition may define locals.

: foo ... [: {: a b :} a b + ;] ... ; \ unambiguous
: bar {: a b :} ... [: a b + ;] ... ; \ ambiguous
: bla {: a ;} ... [: {: b ;} ... b ... ;] ... a ... ; \ unambiguous

The present proposal does not propose that a quotation can access the
locals of a containing definition, because the implementation would be
relatively complex, impose a run-time overhead, and the lifetime of
the locals would be restricted (unless we are willing to accept
further complexity). While there are a number of people who are very
keen to have such a feature, it is not clear how much it would be
used. System implementors are encouraged to provide support for
accessing outer locals on an experimental basis so that programmers
can explore the usefulness of this feature, possibly resulting in a
proposal for this feature in the future.


Syntax
------

The following notations have been seen in the wild, oldest first.
:| ... |;
:[ ... ]:
[: ... ;]

The notation [: ;] appears to have popular support.


Specification
-------------

New system-compilation type quotation-sys with an
implementation-dependent size on the stack. This type contains the
data that needs to be saved for the enclosing colon definition and
restored after the end of the quotation. [Note: we use it in
combination with colon-sys, because DOES> uses colon-sys during
compilation.]

[: bracket-colon
Interpretation: Interpretation semantics for this word are undefined.

Compilation: ( -- quotation-sys colon-sys )

suspends compiling to the current definition, starts a new nested
definition with execution token xt, and compilation continues with
this nested definition. Locals may be defined in the nested
definition. An ambiguous condition exists if a name is used that
satisfies the following constraints: 1) It is not the name of a
currently visible local of the current quotation. 2) It is the name
of a local that was visible right before the start of the present
quotation or any of the containing quotations.

;] semi-bracket
Interpretation: Interpretation semantics for this word are undefined.

Compilation: ( c: quotation-sys colon-sys -- )

Ends the current nested definition, and resumes compilation to the
previous (containing) current definition. It appends the following
run-time to the (containing) current definition:

run-time: ( -- xt )

xt is the execution token of the nested definition.

Add the following ambiguous condition to ";" and ";CODE":

An ambiguous condition exists if the compilation semantics of ";" or
";CODE" are performed inside a quotation.


Implementation
--------------

It is not possible to define quotations in ISO Forth. The following
is an outline definition, where SAVE-DEFINITION-STATE and RESTORE-
DEFINITION-STATE require carnal knowledge of the system and are left
to the implementor.

: [: ( c: -- quotation-sys colon-sys )
postpone ahead save-definition-state :noname ; immediate
: ;] ( c: quotation-sys colon-sys -- ) ( s: -- xt )
postpone ; >r restore-definition-state
postpone then r> postpone literal ; immediate


Compliance Tests
----------------

\ works on Gforth:

T{ : q1 [: 1 ;] ; q1 execute -> 1 }T
T{ : q2 [: [: 2 ;] ;] ; q2 execute execute -> 2 }T
T{ : q3 {: a :} [: {: a b :} b a ;] ; 1 2 3 q3 execute -> 2 1 }T
T{ : q4 [: dup if dup 1- recurse then ;] ; 3 q4 execute .s -> 3 2 1 0 }T
T{ : q5 [: does> drop 4 ;] 5 swap ; create x q5 execute x -> 5 4 }T
T{ : q6 {: a :} [: {: a b :} b a ;] a 1+ ; 1 2 q6 swap execute -> 3 1 }T
T{ 1 2 q6 q6 swap execute execute -> 4 1 }T
T{ 1 2 3 q3 swap q6 swap execute execute -> 3 1 }T

References
----------

Stephen Pelc wrote this; see

http://newsgroups.derkeiler.com/Archive/Comp/comp.lang.forth/2012-07/msg00791.html

I (Alex McDonald) have taken it and modified it as a proposed RfD,
and have removed the discussion of closures. http://www.forth200x.org/

I (Anton Ertl) have taken it and revised it.

The following notes on quotations and closures incorporate material
from several people, including Anton Ertl, Andrew Haley, Marcel
Hendrix, Ward Mcfarland, Bernd Paysan, and Elizabeth Rather. If you
have been left out and want acknowledgement, please let me know.

Additional contributors: Gerry Jackson and others.

[Note]
The phrase "ambiguous condition" has a specific meaning in Forth
standards. An ambiguous condition is "a circumstance for which this
standard does not prescribe a specific behavior". The RfD process
also states: "If you want to leave something open to the system
implementor, make that explicit, e.g., by making it an ambiguous
condition."



VOTING INSTRUCTIONS

Fill out the appropriate ballot(s) below and mail it/them to the
votetaker <an...@mips.complang.tuwien.ac.at>. Your vote will be
published (including your name (without email address) and/or the name
of your system) here. You can vote (or change your vote) at any time
by mailing to me, and the results will be published here.

Note that you can be both a system implementor and a programmer, so
you can submit both kinds of ballots.

Ballot for systems

If you maintain several systems, please mention the systems separately
in the ballot. Insert the system name or version between the brackets
or in the line below the statement. Multiple hits for the same system
are possible (if they do not conflict).

[ ] conforms to ANS Forth.
[ ] already implements the proposal in full since release [ ].
[ ] implements the proposal in full in a development version.
[ ] will implement the proposal in full in release [ ].
[ ] will implement the proposal in full in some future release.
[ ] There are no plans to implement the proposal in full in [ ].
[ ] will never implement the proposal in full.

If you want to provide information on partial implementation, please do
so informally, and I will aggregate this information in some way.


Ballot for programmers

Just mark the statements that are correct for you (e.g., by putting
your name on the line below the statement you agree with). If some
statements are true for some of your programs, but not others, please
mark the statements for the dominating class of programs you write.

[ ] I have used (parts of) this proposal in my programs.
[ ] I would use (parts of) this proposal in my programs if the systems
I am interested in implemented it.
[ ] I would use (parts of) this proposal in my programs if this
proposal was in the Forth standard.
[ ] I would not use (parts of) this proposal in my programs.

If you feel that there is closely related functionality missing from the
proposal (especially if you have used that in your programs), make an
informal comment, and I will collect these, too. Note that the best time
to voice such issues is the RfD stage.

--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2016: http://www.euroforth.org/ef16/

[Matthias Trute]

Vote for amforth and me

> Ballot for systems
>
> [ ] conforms to ANS Forth.
> [X] already implements the proposal in full since release [5.1].

> [ ] implements the proposal in full in a development version.
> [ ] will implement the proposal in full in release [ ].
> [ ] will implement the proposal in full in some future release.
> [ ] There are no plans to implement the proposal in full in [ ].
> [ ] will never implement the proposal in full.
>

> Ballot for programmers
>
> [X] I have used (parts of) this proposal in my programs.
> [X] I would use (parts of) this proposal in my programs if the systems

> I am interested in implemented it.
> [ ] I would use (parts of) this proposal in my programs if this
> proposal was in the Forth standard.
> [ ] I would not use (parts of) this proposal in my programs.
>

Matthias

[Paul Rubin]

an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> Ballot for programmers

> [ ] I have used (parts of) this proposal in my programs.

> [X] I would use (parts of) this proposal in my programs if the systems

> I am interested in implemented it.

> [X] I would use (parts of) this proposal in my programs if this

> proposal was in the Forth standard.
> [ ] I would not use (parts of) this proposal in my programs.

I might post some comments later. Basically as currently proposed it's
more of a minor convenience feature than something that makes Forth
really feel more capable. If they were there I'd use them but I
can do without them.

I'm sympathetic to the calls for quotations to be able to access (at
least for reading) the parent locals, like C++ lambdas (C++'s version of
quotations) can do. I don't think comparisons with Lisp or Scheme are
that useful, but comparisons to C++ might be worthwhile.

The C++ server framework http://www.seastar-project.org/ relies on
lambdas, often nested several levels deep and sometimes using
assignments inside the lambda list, a new feature of C++14.

C++ lambdas like everything else in C++ have a lot of complicated,
interacting features, but all those features address actual use cases
and were deemed worthy of support. They were introduced in C++11 and
enhanced slightly in C++14 and again in C++17 as new uses have appeared,
showing it can take multiple iterations to get something like this
right. I use them regularly in my own C++ code, and someday I want to
try using Seastar (linked above), which relies on them heavily.

[m...@iae.nl]

On Wednesday, August 10, 2016 at 6:54:25 PM UTC+2, Anton Ertl wrote:
> This is actually a poll about how widely the proposal is implemented
> and how popular it is among the programmers. It is called a CfV
> (call-for-votes) because the process is inspired by the Usenet Rdf/CfV
> process.

[..]

> and in the
> iForth case appear to have some significant use and history.

A misunderstanding. Quotations are available as
loadable extension in iForth since April 2013. This does
not mean I use them myself or know of any significant
usage within the iForth community.

We do use ([:) and (;]) as internal words for the iForth
parallel thread wordset. Maybe this is where the remark
comes from?

-marcel

[Gerry Jackson]

On 10/08/2016 17:52, Anton Ertl wrote:
> [X] conforms to ANS Forth.

> [ ] already implements the proposal in full since release [ ].

> [X] implements the proposal in full in a development version.

> [ ] will implement the proposal in full in release [ ].
> [ ] will implement the proposal in full in some future release.
> [ ] There are no plans to implement the proposal in full in [ ].
> [ ] will never implement the proposal in full.
>
> If you want to provide information on partial implementation, please do
> so informally, and I will aggregate this information in some way.
>
>
> Ballot for programmers
>
> Just mark the statements that are correct for you (e.g., by putting
> your name on the line below the statement you agree with). If some
> statements are true for some of your programs, but not others, please
> mark the statements for the dominating class of programs you write.
>

> [X] I have used (parts of) this proposal in my programs.

> [ ] I would use (parts of) this proposal in my programs if the systems
> I am interested in implemented it.
> [ ] I would use (parts of) this proposal in my programs if this
> proposal was in the Forth standard.
> [ ] I would not use (parts of) this proposal in my programs.
>
> If you feel that there is closely related functionality missing from the
> proposal (especially if you have used that in your programs), make an
> informal comment, and I will collect these, too. Note that the best time
> to voice such issues is the RfD stage.

I would like to see [: ... ;] behave like :NONAME ... ; in interpetation
mode

--
Gerry

[Julian Fondren]

On Wednesday, August 10, 2016 at 11:54:25 AM UTC-5, Anton Ertl wrote:
> This is actually a poll about how widely the proposal is implemented
> and how popular it is among the programmers. It is called a CfV
> (call-for-votes) because the process is inspired by the Usenet Rdf/CfV
> process.
>

...

> Rationale
> ---------
>
> There is no situation where quotations cannot be implemented by
> existing Forth words. Specifcially, :NONAME and : provide all the
> features proposed for quotations. Their advantage is as a syntactic
> "sugar" that permits a nameless definition in close proximity to its
> use; and that it avoids generating one-use names only for the purpose
> of refering to the definition inside another word.
>

Code that you print out, frame, and hang on your wall, that's one
thing. But if you're looking at already-written code, usually it is
because you want to change it in some way.

Here's an example from a while back:

: touch-move ( x y -- )
2dup vertical? if
nip height 2/ > if drop-brick else rotate-brick then
else
drop width 2/ > if 0 1 move-brick else 0 -1 move-brick then
then ;

This definition is fine -- except that it's running in a thread, and
is moving bricks simultaneously with another thread moving those
same bricks. So I went ahead and sequenced these interactions with
a semaphore, and since quotations were available, I just wrapped
this whole definition in one:

: touch-move ( x y -- )
[: 2dup vertical? if
nip height 2/ > if drop-brick else rotate-brick then
else
drop width 2/ > if 0 1 move-brick else 0 -1 move-brick then
then ;] in-pit c-section ;

That was a very pleasant change -- because quotations were
available. It wouldn't've been any conceptually harder to use a
once-off definition, but because that is a more annoying change
(though it has the same effect), I might've resisted for a bit. For
example, why not just protect every single usage of this word
instead of putting the protection in the word? Or why not protect
every brick manipulating word instead of these higher level words (I
believe that would've led to some other bugs, but it's been a
while).

It would've been just as easy to write

: touch-move ( x y -- )
2dup vertical? [: if
nip height 2/ > if drop-brick else rotate-brick then
else
drop width 2/ > if 0 1 move-brick else 0 -1 move-brick then
then ;] in-pit c-section ;

Or:

: touch-move ( x y -- )
2dup vertical? if
[: nip height 2/ > if drop-brick else rotate-brick then ;]
else
[: drop width 2/ > if 0 1 move-brick else 0 -1 move-brick then ;]
then in-pit c-section ;

Is either of those better? The point is, I'm more willing to
entertain the question, especially of the last example, because
quotations mean not having to do this:

: touch-move-updown ( y -- )
height 2/ > if drop-brick else rotate-brick then ;

: touch-move-leftright ( x -- )
width 2/ > if 0 1 move-brick else 0 -1 move-brick then ;

: touch-move ( x y -- )
2dup vertical? if
nip ['] touch-move-updown
else
drop ['] touch-move-leftright
then in-pit c-section ;

The original code made it very clear how the touch interface worked.
Well VERTICAL? may not be a great name, but you can see: a touch to
the left moves the brick to the left, and right to the right. A
touch to the top half of the screen drops the brick; a touch to the
bottom half rotates it. VERTICAL? chooses whichever direction is
more extreme -- if your touch is a little bit above the halfway
point, but quite a ways to the right, then your touch is interpreted
to be rightward rather than upward.

A simple interface. Also, maybe not a great one. I may want to
come back to this code later, and make it handle swiping motions
instead. Previously, the entire Forth description of the touch
interface was in a single word, even if that word included 1-2
quotations. In the last example, in a world without quotations, the
logic is split across three words. And not because I thought that
expressed the logic better, but just because I had a bug involving
shared memory.

> Ballot for programmers
...
>
> [X] I have used (parts of) this proposal in my programs.

> [ ] I would use (parts of) this proposal in my programs if the systems
> I am interested in implemented it.
> [ ] I would use (parts of) this proposal in my programs if this
> proposal was in the Forth standard.
> [ ] I would not use (parts of) this proposal in my programs.
>

-- Julian

[Bernd Paysan]

Am Wed, 10 Aug 2016 11:55:14 -0700 schrieb Paul Rubin:
> I'm sympathetic to the calls for quotations to be able to access (at
> least for reading) the parent locals, like C++ lambdas (C++'s version of
> quotations) can do. I don't think comparisons with Lisp or Scheme are
> that useful, but comparisons to C++ might be worthwhile.

C/C++ has no stack the lambda would be able to access. E.g. if you want
to count words in a wordlist with TRAVERSE-WORDLIST and quotations, you do

: count-wl ( wid -- count )
>r 0 [: ( n nt -- n' flag ) drop 1+ true ;] r> traverse-wordlist ;

Now do that in C++, and you need to access an outer local for that. You
don't even have a chance to do it that way, because you can't extend the
stack effect ( nt -- flag ) in a C++ function. In Forth, you can, and
iterators can be written so that they keep their internal state off the
data stack when calling the xt passed to them.

Forth is not a locals-centric language, it's a stack-centric language.
You can use the stack to pass data through quotations used e.g. as
iterators.

Without the stack, quotations would be much less useful.

The problem with nested functions is not only that they are more
difficult to implement, it's also that they can't outlive their parent's
stack frame lifetime, and quotations which do are part of the use cases.

This has been discussed to death already, with Hugh shouting at me for my
"fake quotations". They are Forth quotations, not Lisp or C++ lambdas,
because in Forth, we have stacks. And locals are an optional feature.

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
net2o ID: kQusJzA;7*?t=uy@X}1GWr!+0qqp_Cn176t4(dQ*
http://bernd-paysan.de/

[hughag...@gmail.com]

With the Paysan-faked quotations, locals are not an "optional feature" --- you don't provide any way for the quotation to access the parent function's locals, so locals are not an option at all --- by comparison, with my rquotations the user can either communicate via the data-stack or via locals.

Consider this code in which the rquotation communicates with the parent function via locals:
-----------------------------------------------------------------------------
seq
w field .gender \ 'M' or 'F' \ if someone isn't one or the other, then he/she isn't a person
constant person

: init-person ( str node -- node )
init-seq >r
[char] M r@ .gender ! \ default is male
r> ;

: new-person ( str -- node )
person alloc
init-person ;

person
w field .hotness \ scale of 1 to 10
constant chick

: init-chick ( hotness str node -- node )
init-person >r
[char] F r@ .gender !
r@ .hotness !
r> ;

: new-chick ( hotness str -- node )
chick alloc
init-chick ;

: test { head | people chicks tens nines eights -- }
head r[
1 +to people
dup .gender @ [char] F = if 1 +to chicks
.hotness @
dup 8 = if 1 +to eights then
dup 9 = if 1 +to nines then
10 = if 1 +to tens then \ Yay! We found at least one!
else drop then
]r |each
cr ." total people: " people .
cr ." total chicks: " chicks .
cr ." level 10 chicks: " tens .
cr ." level 9 chicks: " nines .
cr ." level 8 chicks: " eights .
;
-----------------------------------------------------------------------------

Here is a test:
-----------------------------------------------------------------------------
people show-seq
aaaaa
bbbbb
ccccc
Darby
eeeee
fffff
ggggg
hhhhh
ok
people test
total people: 8
total chicks: 5
level 10 chicks: 1
level 9 chicks: 0
level 8 chicks: 2 ok
-----------------------------------------------------------------------------

Now here is a light-weight function in which we don't use locals but rather just communicate via the data-stack (because this time we only have one counter that we are updating):
-----------------------------------------------------------------------------
: chick-counter ( head -- chicks )
0
swap r[ .gender @ [char] F = if 1+ then ]r |each ;
-----------------------------------------------------------------------------

Here is a test:
-----------------------------------------------------------------------------
people chick-counter ok-1
. 5 ok
-----------------------------------------------------------------------------

So, my rquotations are robust --- the user has the option of either communicating with the parent function via locals or via the data-stack --- by comparison, you force the user to always communicate via the data-stack, even when there are several data that need to be communicated (in my first example, the several counters).

All in all, I think that I'm way ahead of Forth-200x --- you aren't even close to having the capability that I have in the novice-package --- this wouldn't bother me, except for the fact that you insist on declaring everything that you do to be the Standard, effectively holding down the entire Forth community to your own level of incompetence.

BTW: I have noticed that every time the subject of quotations and HOFs comes up, you always trot out TRAVERSE-WORDLIST as your example HOF --- I get the impression that TRAVERSE-WORDLIST is the only HOF that you have ever written.

[Julian Fondren]

On Sunday, August 14, 2016 at 4:42:13 PM UTC-5, hughag...@gmail.com wrote:
> With the Paysan-faked quotations, locals are not an "optional feature" ---

Locals are defined in the "The optional Locals word set" of ANS
Forth. They are an "optional feature" to Forth.

>
> BTW: I have noticed that every time the subject of quotations and HOFs comes up, you always trot out TRAVERSE-WORDLIST as your example HOF --- I get the impression that TRAVERSE-WORDLIST is the only HOF that you have ever written.

I can't read this as anything but a request for clf to tell you if
you've worn out your one example and need to come up with another.

[Paul Rubin]

Bernd Paysan <bernd....@gmx.de> writes:
> : count-wl ( wid -- count )
> >r 0 [: ( n nt -- n' flag ) drop 1+ true ;] r> traverse-wordlist ;
> Now do that in C++, and you need to access an outer local for that.

That would be a pretty unnatural interface for traverse-wordlist in C++,
but I see what you mean. It effectively uses the stack as a shared
global. I guess that's ok in this setting. In C++ traverse-wordlist
would probably return an iterator and you'd loop over it or use std::count.

> Forth is not a locals-centric language, it's a stack-centric

> language... Without the stack, quotations would be much less useful.

Locals are optional but I'd hope quotations are even more optional?

> The problem with nested functions is not only that they are more
> difficult to implement, it's also that they can't outlive their parent's
> stack frame lifetime, and quotations which do are part of the use cases.

I wonder how often quotations will leave stuff on the stack for use by
the next iteration of something. I'm used to using a HOF like "fold" or
"reduce" for that, though it would seem weird in Forth.

If you don't need any mutable global state (like the stack) then you can
pass parameters into the lambda, which can survive the parent returning.

[Anton Ertl]

Paul Rubin <no.e...@nospam.invalid> writes:

>Bernd Paysan <bernd....@gmx.de> writes:
>> Forth is not a locals-centric language, it's a stack-centric
>> language... Without the stack, quotations would be much less useful.
>
>Locals are optional but I'd hope quotations are even more optional?

I expect quotations to become CORE EXT, so there would be no
requirement to implement locals in order to implement quotations.
There is also no reason to, given that quotations cannot access outer
locals in the proposed quotations.

>> The problem with nested functions is not only that they are more
>> difficult to implement, it's also that they can't outlive their parent's
>> stack frame lifetime, and quotations which do are part of the use cases.
>
>I wonder how often quotations will leave stuff on the stack for use by
>the next iteration of something. I'm used to using a HOF like "fold" or
>"reduce" for that, though it would seem weird in Forth.

Depending on the HOF, but I expect it to happen pretty often for those
that call the xt multiple times. In Forth, you can use the same HOF
for foreach and for reduce.

>If you don't need any mutable global state (like the stack) then you can
>pass parameters into the lambda, which can survive the parent returning.

You can pass parameters through the stacks in addition to keeping
computed values on the stack. Of course, if there are too many,
things become unwieldy. E.g. (untested):

: fmap ( ... addr u xt -- ... )
\ addr u is a floating-point array with u elements starting at addr.
\ call xt ( ... r -- ... ) once for each element r
... ;

2e 0e v vlen [: ( rexp racc r -- rexp racc1 ) 2 fpick f** f+ :] fmap

Bernd Paysan thinks that's good enough. I am not sure. Especially
when working with several HOFs at once, it becomes hard to follow what
happens on the stack. Or maybe it's just that we are not used to
dealing with that.

- anton

[minf...@arcor.de]

I am speaking for MinForth which is non-public (not to confound with its proof-of-concept ancestor published nearly 2 decades ago)

IMO quotations are not worth their syntactic charme, when SW maintenance will not be done by the original programmer himself.

Ballot for systems
[X] conforms to ANS Forth.

[ ] already implements the proposal in full since release [ ].
[ ] implements the proposal in full in a development version.
[ ] will implement the proposal in full in release [ ].
[ ] will implement the proposal in full in some future release.
[ ] There are no plans to implement the proposal in full in [ ].

[X] will never implement the proposal in full.

Ballot for programmers

[ ] I have used (parts of) this proposal in my programs.
[ ] I would use (parts of) this proposal in my programs if the systems
I am interested in implemented it.
[ ] I would use (parts of) this proposal in my programs if this
proposal was in the Forth standard.

[X] I would not use (parts of) this proposal in my programs.

[Paul Rubin]

an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> I expect quotations to become CORE EXT

Hmm, that surprises me a bit, since quotations seem like an esoteric
feature for most Forth users. But I guess the implementation can be
very simple..

> so there would be no requirement to implement locals in order to
> implement quotations.

IMO locals are much more useful, so regardless of implementation
dependencies, if I had to pick just one I'd pick locals.

> 2e 0e v vlen [: ( rexp racc r -- rexp racc1 ) 2 fpick f** f+ :] fmap

> Bernd Paysan thinks that's good enough. I am not sure. Especially
> when working with several HOFs at once, it becomes hard to follow what
> happens on the stack. Or maybe it's just that we are not used to
> dealing with that.

That example is readable but yes, it does get worse with deeper levels
of HOF. Witness Haskell vs Python: the languages have similar raw
capabilities, but Haskell uses HOFs more freely because its type system
keeps them from getting out of control.

I think that Forthers generally don't want to use so much abstraction so
maybe it won't be an issue. In fact I wonder if we language geeks are
irresistably drawn to features that working practitioners (hardware
designers etc.) don't particularly care about. The whole quotation
discussion could be an example of that.

[Anton Ertl]

Paul Rubin <no.e...@nospam.invalid> writes:
>an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> I expect quotations to become CORE EXT
>
>Hmm, that surprises me a bit, since quotations seem like an esoteric
>feature for most Forth users.

CORE EXT is the typical place for things that do not belong anywhere
else. Another place might be TOOLS EXT, but looking at 15.1 and
15.6.1, I don't think that's the right place (but the same could be
said of a number of things in TOOLS EXT; e.g., AHEAD, CODE, ;CODE
etc. are not just used during development, but also in the finished
applications.

Anyway, if we put it in TOOLS EXT, you can still have a standard
system with quotations and without locals.

>> 2e 0e v vlen [: ( rexp racc r -- rexp racc1 ) 2 fpick f** f+ :] fmap
>
>> Bernd Paysan thinks that's good enough. I am not sure. Especially
>> when working with several HOFs at once, it becomes hard to follow what
>> happens on the stack. Or maybe it's just that we are not used to
>> dealing with that.
>
>That example is readable but yes, it does get worse with deeper levels
>of HOF. Witness Haskell vs Python: the languages have similar raw
>capabilities, but Haskell uses HOFs more freely because its type system
>keeps them from getting out of control.

I think that the tendency of Haskell to use currying (coming from the
syntax) plays a bigger role, as well as the culture of functional
programming.

>I think that Forthers generally don't want to use so much abstraction so
>maybe it won't be an issue. In fact I wonder if we language geeks are
>irresistably drawn to features that working practitioners (hardware
>designers etc.) don't particularly care about. The whole quotation
>discussion could be an example of that.

Certainly, looking at the current CfV results, there seem to be a
bunch of people who have already used quotations, and a bunch of
people who say they will never use quotations. I have worked around
their absence (by defining helper words) for decades, other people
have worked around their absence with return-address maniupulations,
so I don't think that practitioners don't care about them, unless you
define a practitioner to be someone who does not care about
quotations.

Looking at other languages, closures have been limited to functional
languages for a long time, but a few years ago, "lambdas" started
appearing in C#, then in Java and C++. I guess there's a contingent
of programmers in these languages who don't need lambdas and won't use
it, but others use lambdas; it's probably too early to tell the
overall effect of lambda on these languages.

An earlier innovation was genericity; at the start this was probably
also something only for language geeks, but I think it's now
mainstream in Java, C++, and C#. Just if anybody wonders if we need
this in Forth: Genericity is a solution for a problem that these
languages get from their static typechecking; Forth has no
typechecking, so it does not need to add genericity to the type
system.

[Bernd Paysan]

Am Tue, 16 Aug 2016 07:13:57 +0000 schrieb Anton Ertl:
> : fmap ( ... addr u xt -- ... )
> \ addr u is a floating-point array with u elements starting at addr. \
> call xt ( ... r -- ... ) once for each element r
> ... ;
>
> 2e 0e v vlen [: ( rexp racc r -- rexp racc1 ) 2 fpick f** f+ :] fmap
>
> Bernd Paysan thinks that's good enough. I am not sure. Especially when
> working with several HOFs at once, it becomes hard to follow what
> happens on the stack. Or maybe it's just that we are not used to
> dealing with that.

As usual in Forth, you shouldn't deal with "too many variables" (see
Thinking Forth page 210 in the online edition), nor should you write a
word with HOFs inside HOFs inside HOFs.

If you really need a complex data structure, use and OOP package, best
one with a current object, because that current object will be current
inside the quotation, too, without any stack juggling.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:
>Am Tue, 16 Aug 2016 07:13:57 +0000 schrieb Anton Ertl:
>> Bernd Paysan thinks that's good enough. I am not sure. Especially when
>> working with several HOFs at once, it becomes hard to follow what
>> happens on the stack. Or maybe it's just that we are not used to
>> dealing with that.
>
>As usual in Forth, you shouldn't deal with "too many variables" (see
>Thinking Forth page 210 in the online edition),

This talks about global variables, and says nothing about "too many
variables". Maybe you mean the cartoon on Page 209, but that does not
present a solution to the problem.

>nor should you write a
>word with HOFs inside HOFs inside HOFs.

HOFs inside HOFs are easy. There is only one HOF to consider at each
point in this case. It's harder if you pass HOFs to HOFs.

Anyway, if a Forther gives me "you should not"s without a supporting
argument, nor presenting an alternative approach, it's an indication
that there is some lack in the language. So maybe keeping the
not-quite-local data on the data stack is not enough to make accesses
to outer locals unnecessary.

>If you really need a complex data structure, use and OOP package, best
>one with a current object, because that current object will be current
>inside the quotation, too, without any stack juggling.

Ok, so your solution is to have one global variable, and access all
the other data through that. I very much doubt that is sufficient,
because global variables are not sufficient to implement nested
Pascal's and GNU C's nested functions, either; instead, they do static
link chains and trampolines. E.g, I don't think you can do the
man-or-boy-test with that.

[hughag...@gmail.com]

On Sunday, August 21, 2016 at 2:57:30 AM UTC-7, Anton Ertl wrote:

> Bernd Paysan <bernd....@gmx.de> writes:
> >nor should you write a
> >word with HOFs inside HOFs inside HOFs.
>
> HOFs inside HOFs are easy. There is only one HOF to consider at each
> point in this case. It's harder if you pass HOFs to HOFs.

The phrase "HOFs inside HOFs" makes no sense --- you can have quotations inside of functions, but HOFs are named functions which are not inside of other functions --- I've noticed before that you guys' terminology is mixed up; you seem to not know the difference between a HOF and a quotation.

These are my rules for rquotations:
1.) The rquotation must be called with REX rather than EXECUTE.
2.) Only a HOF can use REX.
3.) A HOF can not call another HOF.
4.) A HOF must have local variables.

If you do this, then the rquotation can access the parent function's local variables.

These are easy rules to follow because this is the way that you would want to use quotations almost all of the time anyway.

[Bernd Paysan]

Am Sun, 21 Aug 2016 09:32:58 +0000 schrieb Anton Ertl:

> Ok, so your solution is to have one global variable, and access all the
> other data through that. I very much doubt that is sufficient, because
> global variables are not sufficient to implement nested Pascal's and GNU
> C's nested functions, either; instead, they do static link chains and
> trampolines. E.g, I don't think you can do the man-or-boy-test with
> that.

Well, the boy-or-man test isn't really needing better quotations, it's
needing a bit more OOP. It wants something applied in a "context", and
we call that "method". While you could have many methods, here, one is
sufficient. While you could have many different object lifetime schemes,
stack-allocated here is sufficient.

require mini-oof2.fs

\ define a class for the default values

object class
field: k
method b
end-class mob

:noname ( -- n ) k @ ; mob to b
mob new constant m-1 -1 m-1 .k !
mob new constant m0 0 m0 .k !
mob new constant m1 1 m1 .k !

\ define a class for the context

mob class
field: x1
field: x2
field: x3
field: x4
field: x5
end-class b-ctx

: a ( k x1 x2 x3 x4 x5 -- result )
b-ctx new >o
x5 ! x4 ! x3 ! x2 ! x1 ! k !
k @ 0<= IF
x4 @ .b x5 @ .b +
ELSE
b
THEN
dispose o> ;

:noname ( -- n )
-1 k +!
k @ o x1 @ x2 @ x3 @ x4 @ a
; b-ctx to b

: a-n ( n -- n' )
m1 m-1 m-1 m1 m0 a ;

Test with a bit more return stack space than usual (I just set it to 1G):

27 0 [do] cr [i] dup . a-n . [lOOP]
0 1
1 0
2 -2
3 0
4 1
5 0
6 1
7 -1
8 -10
9 -30
10 -67
11 -138
12 -291
13 -642
14 -1446
15 -3250
16 -7244
17 -16065
18 -35601
19 -78985
20 -175416
21 -389695
22 -865609
23 -1922362
24 -4268854
25 -9479595
26 -21051458 ok

Looks good. If you want OOP, use OOP. If you want something that is
slightly less than OOP, just use OOP, too, please. Unlike nested
functions, OOP is dead easy to implement. It really is powerful enough,
way more powerful than nested functions, as powerful as closures. As
quotations are mostly syntactic sugar, they are not needed at all here.

Maybe you want an object on the locals stack to get rid of the relatively
expensive new/dispose pairs.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:
>Am Sun, 21 Aug 2016 09:32:58 +0000 schrieb Anton Ertl:
>
>> Ok, so your solution is to have one global variable, and access all the
>> other data through that. I very much doubt that is sufficient, because
>> global variables are not sufficient to implement nested Pascal's and GNU
>> C's nested functions, either; instead, they do static link chains and
>> trampolines. E.g, I don't think you can do the man-or-boy-test with
>> that.
>
>Well, the boy-or-man test isn't really needing better quotations, it's
>needing a bit more OOP.

Sure, you can implement the functionality in some way in any
Touring-complete language. But the original test was for whether
nested functions in Algol 60 were correctly implemented. So if it can
be implemented with closures (actually Pascal function parameters are
sufficient), but not with the proposed quotations, it shows that
passing data on the stack does not replace non-local data accesses in
every case. Of course, as is often the case in Forth, the question is
if the various approaches for avoiding a feature are sufficient to
make that feature unnecessary or not.

For comparsion, here's the Algol 60 variant:

begin
real procedure A(k, x1, x2, x3, x4, x5);
value k; integer k;
begin
real procedure B;
begin k := k - 1;
B := A := A(k, B, x1, x2, x3, x4);
end;
if k <= 0 then A := x4 + x5 else B;
end;
outreal(A(10, 1, -1, -1, 1, 0));
end;

and my attempt at a version in Forth with access to outer locals,
based on <http://rosettacode.org/wiki/Man_or_boy_test#Pascal> (my
Algol 60 is not good enough to grasp the fine details of the Algol 60
version):

0 constant zero
1 constant one
-1 constant negone

: A {: k x1 x2 x3 x4 x5 | B -- n :} recursive
[: k 1- to k
k B x1 x2 x3 x4 A ;] to B
k 0 <= if
x4 execute x5 execute +
else
B execute
then ;

10 ' one ' negone ' negone ' one ' zero A .

So access to outer locals makes this quite a bit shorter (14 instead
of 41 lines) than making the frames explicit by coding them in OOP.

The question is how often that helps for real-world programs. My
guess is that, with the Pascal-like restriction, not often enough to
make it worthwhile. With garbage-collection of closures, it would
probably be worthwhile, but we probably don't want to go there; and
for something in between, we will have to see.

[Bernd Paysan]

Am Mon, 22 Aug 2016 13:54:32 +0000 schrieb Anton Ertl:
> The question is how often that helps for real-world programs. My guess
> is that, with the Pascal-like restriction, not often enough to make it
> worthwhile. With garbage-collection of closures, it would probably be
> worthwhile, but we probably don't want to go there; and for something in
> between, we will have to see.

MINOS uses a lot of closure-like objects+quotations, the "actions". An
action is bound to a widget, contains a state variable of its own (e.g.
on/off for a toggle button), and an action (or two), stored as quotation
xt; the quotation is performed in the context of the object that created
those actions and widgets. Since all that commons stuff is reused again
and again, it's just a slightly different version of [: ;] around a block
of code to instantiate them. So the line count is probably not a problem.

In Common Lisp, you would do that with closures. In an OOP system, you
do it with OOP. And if you want a more compact format of declaring
things, just define them. You could have an OOP system that declares the
class in one line, and creates you an instantiation code that takes the
initial values from stack, just with about the same amount of code we
need to implement locals (a lot more code than mini-oof2.fs).

<base class> O{ k x1 x2 x3 x4 x5 }O mob

certainly is possible. There's even a mini-oof.fs based implementation
of closures, which does just that. If we wanted to generalize the locals
declaration, we could easily reuse most of the code in glocals.fs; after
all, the declaration of a set of locals and the declaration of a set of
instance variables is the same problem: Create a bunch of names which
address consecutive elements in memory; one time on a locals stack,
another time in an object. And Algol 60 even recognizes that by calling
the locals "activation record".

If you look at the Rosetta code page of the man-or-boy test, you'll find
a number of implementations which use classes to implement the activation
record, so I think that approach is fair and useful.

http://rosettacode.org/wiki/Man_or_boy_test

If you want a class creation syntax which reduces line count, it's also
doable. mini-oof2.fs just shares code with the structure declaration,
sharing code with the locals declaration would also be possible.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:
>In Common Lisp, you would do that with closures. In an OOP system, you
>do it with OOP.

Somehow, in the last few years, several popular object-oriented
languages, e.g., Java, C++, and C#, have acquired lambdas; apparently
not everybody does it with OOP.

And actually you are not using the distinctive feature of OOP: dynamic
binding. What you use, is the current-object pointer. However, this
allows access only to locals from one activation record; that's enough
for the man-or-boy test, but not in the general case. Is the general
case needed? Yes, in particular in curried functions each argument is
in a separate activation record.

>And if you want a more compact format of declaring
>things, just define them. You could have an OOP system that declares the
>class in one line, and creates you an instantiation code that takes the
>initial values from stack, just with about the same amount of code we
>need to implement locals (a lot more code than mini-oof2.fs).
>
><base class> O{ k x1 x2 x3 x4 x5 }O mob
>
>certainly is possible. There's even a mini-oof.fs based implementation
>of closures, which does just that. If we wanted to generalize the locals
>declaration, we could easily reuse most of the code in glocals.fs; after
>all, the declaration of a set of locals and the declaration of a set of
>instance variables is the same problem: Create a bunch of names which
>address consecutive elements in memory; one time on a locals stack,
>another time in an object.

Is the locals-derived "class" definition useful for anything besides
locals?

Of course, if we support non-local locals, we will have to enhance the
locals code to create activation records when needed, and also to
create the access to the right activation record. The definition part
is similar to a structure or class definition, but the access part is
quite different for the general case.

We also need a way to specify how an activation record is allocated
and how long it lives. E.g., for something CREATE-DOES>-like we want
to ALLOT the memory:

: constant ( x "name" -- )
['] here-allot <{: x :}
[: x :] <create ;

This would allot the activation record (and HERE-ALLOT is an ALLOT
that returns the address of the allotted memory), and at run-time it
accesses the local and return it.

>If you look at the Rosetta code page of the man-or-boy test, you'll find
>a number of implementations which use classes to implement the activation
>record, so I think that approach is fair and useful.

I also find that the C++11 and C# version 3+ have shorter source code
by using anonymous functions/lambda expressions.

[Anton Ertl]

hughag...@gmail.com writes:
>On Sunday, August 21, 2016 at 2:57:30 AM UTC-7, Anton Ertl wrote:
>> Bernd Paysan <bernd....@gmx.de> writes:

>> >nor should you write a=20

>> >word with HOFs inside HOFs inside HOFs.

>>=20

>> HOFs inside HOFs are easy. There is only one HOF to consider at each
>> point in this case. It's harder if you pass HOFs to HOFs.
>
>The phrase "HOFs inside HOFs" makes no sense

What I meant is a call of a higher-order word inside a higher-order
word. A typical example is

: with-base ( n xt -- )
base @ >r
[: swap base ! execute ;] catch
r> base ! throw ;

12345 $10 ' . with-base

WITH-BASE takes and xt and is therefore a higher-order word, and so is
CATCH.

[Bernd Paysan]

Am Wed, 24 Aug 2016 15:54:06 +0000 schrieb Anton Ertl:

> And actually you are not using the distinctive feature of OOP: dynamic
> binding. What you use, is the current-object pointer. However, this
> allows access only to locals from one activation record; that's enough
> for the man-or-boy test, but not in the general case. Is the general
> case needed? Yes, in particular in curried functions each argument is
> in a separate activation record.

What about this part of my source code was not clear to you?

object class
field: k
method b
end-class mob

:noname ( -- n ) k @ ; mob to b \ bind xt to b of mob

...

:noname ( -- n )
-1 k +!
k @ o x1 @ x2 @ x3 @ x4 @ a

; b-ctx to b \ bind xt to b of b-ctx

Method b is dynamically bound to either work for the constants or for the
recursive call.

Note that the lambda closures in C++ and Java are syntactic sugar around
classes, they aren't implemented the same way as you would do it in Algol
60. C++ overloads the function call to make it happen, which actually
means that you can't pass a lambda closure to a function that takes a
normal function pointer.

Only lambdas without any captured variables are compatible with function
pointers, all others need std::function as type.

http://www.cprogramming.com/c++11/c++11-lambda-closures.html

In Java, it was easier, as Java didn't have any function pointer type to
start with. Now it has one, and it's just calling a specified method of
a particular class, something Java's VM can do.

[Anton Ertl]

Bernd Paysan <be...@net2o.de> writes:
>Am Wed, 24 Aug 2016 15:54:06 +0000 schrieb Anton Ertl:
>
>> And actually you are not using the distinctive feature of OOP: dynamic
>> binding. What you use, is the current-object pointer. However, this
>> allows access only to locals from one activation record; that's enough
>> for the man-or-boy test, but not in the general case. Is the general
>> case needed? Yes, in particular in curried functions each argument is
>> in a separate activation record.
>
>What about this part of my source code was not clear to you?

What makes you think that it is not clear to me?

>Note that the lambda closures in C++ and Java are syntactic sugar around
>classes, they aren't implemented the same way as you would do it in Algol
>60.

Earlier you argued that activation records and objects are the same thing.

I see two differences:

* Objects are connected to dynamic binding of methods, something that
is not needed for activation records. But that extra capability
does not hurt when implementing activation records with objects.

* Object-oriented languages normally have only one current object
(this or self), while there several activation records can be active
at the same time, one for each outer function that has local
variables. That is what I was pointing out above.

As a minimal example, consider a curried implementation
of ROT in a dialect of Forth with full closures:

: crot { x } [: { y } [: { z } y x z ;] ;] ;
1 2 3 crot execute execute \ now on the stack: 2 3 1

Here you have an activation record containing X, and another
activation record containing Y, and you cannot keep both of those in
the current object, but need to implement some additional mechanism
to perform the access; in cases with read-only locals like this one,
you can shift the additional work to the definition (by also
defining a copy of X in the Y activation record, and maybe copies of
X and Y in the Z activation record).

But in cases with locals that are changed, you cannot do that. The
classical implementations are to use a static link chain or a
display, but the C++ syntax suggests keeping the addresses of the
accessed variables in the later activation records (one can also see
this as a variation on the display theme). Anyway, all of this
requires something that's a bit different from an ordinary
object-oriented field access.

>C++ overloads the function call to make it happen, which actually
>means that you can't pass a lambda closure to a function that takes a
>normal function pointer.
>
>Only lambdas without any captured variables are compatible with function
>pointers, all others need std::function as type.
>
>http://www.cprogramming.com/c++11/c++11-lambda-closures.html

My guess is that this difference between ordinary function pointers
and std::function is that ordinary function pointers are C-compatible
and don't come with an environment, while std:function includes an
environment. They apparently did not want to go for trampolines, and,
after all they are C++: Introducing yet another type and then trying
to hide it with overloading is their way of life.

>In Java, it was easier, as Java didn't have any function pointer type to
>start with. Now it has one, and it's just calling a specified method of
>a particular class, something Java's VM can do.

The hard part for Java apparently was to make it fit in the type
system (at least my colleague who is into type systems tells me that).
Fortunately we don't have that problem in Forth.

Anyway, both languages have added lambdas, however implemented,
because they allow expressing programs more succinctly than with just
the object-oriented features they had before.

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>
>>In Java, it was easier, as Java didn't have any function pointer type to
>>start with. Now it has one, and it's just calling a specified method of
>>a particular class, something Java's VM can do.
>
> The hard part for Java apparently was to make it fit in the type
> system (at least my colleague who is into type systems tells me
> that).
> Fortunately we don't have that problem in Forth.

I'm not sure what problem that might have been. This is a declaration
of a type "function int -> int":

interface IntToIntFunction {
int apply(int n);
}

and here's the declaration of a function of that type:

IntToIntFunction square = (x -> x * x);

This declaration is the same as:

IntToIntFunction square = new IntToIntFunction() {
public int apply(int n) { return n * n; }
};

No change to the Java type system was needed.

I don't think that this decision was very controversial in the end
because there were already Java libraries which used this kind of
interface and nobody wanted to write new "lambda-ified" libraries.
And, of course, this was easy for Java programmers to understand.

Andrew.

[Anton Ertl]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>
>>>In Java, it was easier, as Java didn't have any function pointer type to
>>>start with. Now it has one, and it's just calling a specified method of
>>>a particular class, something Java's VM can do.
>>
>> The hard part for Java apparently was to make it fit in the type
>> system (at least my colleague who is into type systems tells me
>> that).
>> Fortunately we don't have that problem in Forth.
>
>I'm not sure what problem that might have been.

Sorry, I am not into this, so I cannot explain it to you.

However, even arrays of objects have often given me type trouble in
Java, and he explained that this had to do with IIRC
co/contravariance, so I can easily believe that designing lambdas in a
way that fits with Javas type system was a non-trivial exercise; after
all, java takes type checking seriously, and does not have the C++
escape hatch of declaring undefined behaviour when the going gets
tough. Of course, once they had such a design, it just works.

[Gerry Jackson]

On 10/08/2016 17:52, Anton Ertl wrote:

> Specification
> -------------
>
> New system-compilation type quotation-sys with an
> implementation-dependent size on the stack. This type contains the
> data that needs to be saved for the enclosing colon definition and
> restored after the end of the quotation. [Note: we use it in
> combination with colon-sys, because DOES> uses colon-sys during
> compilation.]
>
> [: bracket-colon
> Interpretation: Interpretation semantics for this word are undefined.
>
> Compilation: ( -- quotation-sys colon-sys )

shouldn't this be
Compilation: ( C: -- quotation-sys )
Why mention the colon-sys? There may be other control-flow items on the
control-flow stack between the outer definition, which may be a colon
definition or a quotation, and the quotation-sys. Similarly the
specification for ;]

>
> suspends compiling to the current definition, starts a new nested
> definition with execution token xt, and compilation continues with
^^^^^^^^^^^^^^^^^^
An execution token is not mentioned above, so why mention it here. It is
sufficient and better to leave it as quotation-sys which may, of course,
contain the xt. If an xt was specified, as with :NONAME, my system would
have a problem with xt values with nested quotations e.g.

: foo [: ... [: ( -- xt1 ) .... ;] ( -- xt2 ) ... ;] ...;

because during compilation xt1 = xt2 because no executable code has been
saved, the xt's are made correct when the terminating ;]'s are executed.

> this nested definition. Locals may be defined in the nested
> definition. An ambiguous condition exists if a name is used that
> satisfies the following constraints: 1) It is not the name of a
> currently visible local of the current quotation. 2) It is the name
> of a local that was visible right before the start of the present
> quotation or any of the containing quotations.

I don't see a good reason why the specification for :NONAME states that
it leaves an xt on the data stack as that constrains an implementation -
it would be better to leave a noname-sys on the control stack. It is not
easy for a user to access the xt during compilation so why not leave it
open for system implementers. I realise it would slightly complicate the
specification of ; because that would have to be changed to leave an xt
at the end of a :NONAME definition.

--
Gerry

[Anton Ertl]

Gerry Jackson <ge...@jackson9000.fsnet.co.uk> writes:
>On 10/08/2016 17:52, Anton Ertl wrote:
>> Specification
>> -------------
>>
>> New system-compilation type quotation-sys with an
>> implementation-dependent size on the stack. This type contains the
>> data that needs to be saved for the enclosing colon definition and
>> restored after the end of the quotation. [Note: we use it in
>> combination with colon-sys, because DOES> uses colon-sys during
>> compilation.]
>>
>> [: bracket-colon
>> Interpretation: Interpretation semantics for this word are undefined.
>>
>> Compilation: ( -- quotation-sys colon-sys )
>
>shouldn't this be
> Compilation: ( C: -- quotation-sys )
>Why mention the colon-sys?

As mentioned, above, because of DOES>. I.e., the CfV proposes that

[: ... does> ... ;]

works, and because DOES> consumes a colon-sys and produces another
one, we have to have a colon-sys on the stack.

As for the C:, it is as redundant as the F:. Control-flow stack items
always go on the control-flow stack, and floating-point values always
go on the FP stack.

> There may be other control-flow items on the
>control-flow stack between the outer definition, which may be a colon
>definition or a quotation, and the quotation-sys.

Sure, but they are below the quotation-sys, are not changed by [: or
;], and therefore don't play a role here.

>> suspends compiling to the current definition, starts a new nested
>> definition with execution token xt, and compilation continues with
> ^^^^^^^^^^^^^^^^^^
>An execution token is not mentioned above, so why mention it here.

To make it clear that this is the xt that is referred to below, in the
run-time semantics of ;]. Maybe that is not the best way to write it,
and maybe we will find a better way in the wordsmithing that's usually
done at the meeting, but I hope that's clear enough for CfV purposes.
However, if you have a problem with the way this is written, it's good
to point that out now, so that the chances are better that the
committee will not miss the point.

> It is
>sufficient and better to leave it as quotation-sys which may, of course,
>contain the xt. If an xt was specified, as with :NONAME, my system would
>have a problem with xt values with nested quotations e.g.
>
>: foo [: ... [: ( -- xt1 ) .... ;] ( -- xt2 ) ... ;] ...;
>
>because during compilation xt1 = xt2 because no executable code has been
>saved, the xt's are made correct when the terminating ;]'s are executed.

Actually the way it is specified does not make the xt explicit on the
stack, so that would not be a problem for your system.

>I don't see a good reason why the specification for :NONAME states that
>it leaves an xt on the data stack as that constrains an implementation -
>it would be better to leave a noname-sys on the control stack. It is not
>easy for a user to access the xt during compilation so why not leave it
>open for system implementers. I realise it would slightly complicate the
>specification of ; because that would have to be changed to leave an xt
>at the end of a :NONAME definition.

So here you have the answer.

Of course, as you point out, programs have a hard time to make access
of that xt before the ";", and are therefore unlikely to do that, so
you could declare an environmental restriction that disallows such
usage, and probably you would still be able to run most standard
programs that are out there.

[Gerry Jackson]

Sorry my mistake, I misunderstood something.

>
>>> suspends compiling to the current definition, starts a new nested
>>> definition with execution token xt, and compilation continues with
>> ^^^^^^^^^^^^^^^^^^
>> An execution token is not mentioned above, so why mention it here.
>
> To make it clear that this is the xt that is referred to below, in the
> run-time semantics of ;]. Maybe that is not the best way to write it,
> and maybe we will find a better way in the wordsmithing that's usually
> done at the meeting, but I hope that's clear enough for CfV purposes.
> However, if you have a problem with the way this is written, it's good
> to point that out now, so that the chances are better that the
> committee will not miss the point.
>
>> It is
>> sufficient and better to leave it as quotation-sys which may, of course,
>> contain the xt. If an xt was specified, as with :NONAME, my system would
>> have a problem with xt values with nested quotations e.g.
>>
>> : foo [: ... [: ( -- xt1 ) .... ;] ( -- xt2 ) ... ;] ...;
>>
>> because during compilation xt1 = xt2 because no executable code has been
>> saved, the xt's are made correct when the terminating ;]'s are executed.
>
> Actually the way it is specified does not make the xt explicit on the
> stack, so that would not be a problem for your system.

Yes I realised that, I just don't want an xt introduced in the stack
diagram of [:. Perhaps just add a few words to say the run-time xt given
by ;].

>
>> I don't see a good reason why the specification for :NONAME states that
>> it leaves an xt on the data stack as that constrains an implementation -
>> it would be better to leave a noname-sys on the control stack. It is not
>> easy for a user to access the xt during compilation so why not leave it
>> open for system implementers. I realise it would slightly complicate the
>> specification of ; because that would have to be changed to leave an xt
>> at the end of a :NONAME definition.
>
> So here you have the answer.

It's not very difficult to change ; to only leave an xt if terminating a
:NONAME definition with an optional intervening DOES>

>
> Of course, as you point out, programs have a hard time to make access
> of that xt before the ";", and are therefore unlikely to do that, so
> you could declare an environmental restriction that disallows such
> usage, and probably you would still be able to run most standard
> programs that are out there.
>
> - anton
>


--

Gerry

[Anton Ertl]

Gerry Jackson <ge...@jackson9000.fsnet.co.uk> writes:
>On 31/08/2016 16:26, Anton Ertl wrote:
>> Actually the way it is specified does not make the xt explicit on the
>> stack, so that would not be a problem for your system.
>
>Yes I realised that, I just don't want an xt introduced in the stack
>diagram of [:. Perhaps just add a few words to say the run-time xt given
>by ;].

It is not in the stack diagram of "[:". I'll present your concern if
somebody proposes adding it to the stack diagram of "[:" in the
committee (but I think that is unlikely).

>>> I don't see a good reason why the specification for :NONAME states that
>>> it leaves an xt on the data stack as that constrains an implementation -
>>> it would be better to leave a noname-sys on the control stack. It is not
>>> easy for a user to access the xt during compilation so why not leave it
>>> open for system implementers. I realise it would slightly complicate the
>>> specification of ; because that would have to be changed to leave an xt
>>> at the end of a :NONAME definition.
>>
>> So here you have the answer.
>
>It's not very difficult to change ; to only leave an xt if terminating a
>:NONAME definition with an optional intervening DOES>

It may not be difficult to change in your private copy, but changing
it in the standard is a bit more involved. It starts with a proposal,
and nobody has written one yet. Also, in cases like this where a
feature that programs might rely on would be de-standardized, it takes
one standard period where the feature is still in the standard, but
obsolescent.