Recognizer proposal; the action set

[Alex McDonald]

(email also posted to forth200x group)

The following 3 parts make up the recognizer.

1. Action-set
A 3 cell sized contiguous area, where each cell is an execution token
for an action. The first is executed during interpretation; the second
during compilation; and the third while POSTPONEing.

2. Recognizer
A string parsing word that returns converted values and an action-set.

3. Recognizer Sequence
An ordered set of recognizers.


This is a discussion about the action-set.

The action set drives each of the individual execution tokens of what to
do when interpreting I, compiling C and postponing P. We can define a
system with 3 actions, one for each of I C and P, or we can specify 2
actions I and C, and define by default the P action. (The order of the
cells is immaterial, so here we define the order of 3 cells as I C P and
the action-set is the address of I.)

3 cell action-set, explicit POSTPONE

: NOOP ;
: PLIT POSTPONE LITERAL ;
: PPLIT PLIT POSTPONE PLIT ;
CREATE ACT-NUM \ 3 cell area
‘ NOOP , \ interpret action
‘ PLIT , \ compile action
‘ PPLIT , \ postpone action

: POSTPONE ( "name" -- )
PARSE-NAME FORTH-RECOGNIZER RECOGNIZE
CELL+ CELL+ @ EXECUTE ;

Considerations.

1. Is the concept and hence name of an action-set acceptable?

2. Are there to be further discussions on the 2 cell auto-magic POSTPONE
action, or does Stephen Pelc's comment "We cannot predict how
recognisers will be used, so attempting to automate the POSTPONE actions
is doomed to failure" stand?

3. Since the sequence of the cells is defined, is there any requirement
to have words such as

: ACT>INT @ ;
: ACT>COMP CELL+ @ ;
: ACT>POST CELL+ CELL+ @ ;

(These were previously RECTYPE> or recognizer types, rather than
action-sets)

The only reason for their existence appears to be for use in POSTPONE
(see above) or the interpreter where

( action-set ) STATE @ IF ACT>COMP ELSE ACT>INT THEN EXECUTE

can be more succinctly written as

( action-set ) STATE @ IF CELL+ THEN @ EXECUTE


--
Alex

[Anton Ertl]

Alex McDonald <al...@rivadpm.com> writes:
>2. Are there to be further discussions on the 2 cell auto-magic POSTPONE
>action, or does Stephen Pelc's comment "We cannot predict how
>recognisers will be used, so attempting to automate the POSTPONE actions
>is doomed to failure" stand?

Reference?

In any case, I described the necessity for various actions in
<2020May2...@mips.complang.tuwien.ac.at> (referring to RfC-D
actions) in an answer to a posting by you.

|If you just want to recognize literals, you need just one action: The
|time-shifting action like POSTPONE LITERAL (that's coming out of
|RECTYPE>POST); you don't do anything to the literal (that the
|recognizer left, e.g., on the data stack) interpretively, you
|time-shift it once for compilation, and you time-shift it twice for
|postponing. SwiftForth has a recognizer-like extension mechanism that
|provides a single xt: that for the time-shifting action.
|
|If you want a recognizer for normal words, or, e.g., for TO
|replacement syntax ->X, you als need a run-time action; for normal
|words the run-time action is EXECUTE, for ->X, where X is a VALUE, it
|is "!". I.e., after time-shifting the xt (for a word recognizer) or
|address (for a ->X recognizer) 0-2 times, you perform the run-time
|action (also appropriately time-shifted). In the proposal we have a
|one-time-shifted run-time action in the form of RECTYPE>COMP (for
|literals the result is the same action as the time-shifting action).
|
|If you want a recognizer that can also handle immediate and combined
|words, these two actions are insufficient. For a combined word, the
|interpretation semantics is not systematically related to the
|compilation semantics, so it needs a separate action. And that's what
|RECTYPE>INT provides. For literals it is noop, for ->X it's the
|run-time action without time-shift (i.e., "!" for a VALUE), for words
|(both normal and combined) it's performing the interpretation
|semantics (i.e., for an nt-based word recognizer the action consists
|essentially of NAME>INTERPRET EXECUTE); for combined words the
|RECTYPE>COMP action performs the compilation semantics.

If you have any trouble understanding that, please ask.

So yes, you need three actions. The only question is whether
postponing is performed by 1) time-shifting the data followed by
time-shifting the compilation action (RFC-D), or whether 2) you
combine these in one action (IIRC RFC-C). Option 1 is a little harder
to understand, but requires less code for many rectypes, and I think
it is also less error-prone because of that. Here are some rectype
definitions that I find in Gforth's code (i.e., an option-1 system):

' noop ' fliteral ' fliteral rectype: rectype-float
' getenv ' env$, ' slit, rectype: rectype-env
' noop ' slit, dup rectype: rectype-string
' (to) ' (to), ' post-to, rectype: rectype-to
' >system ' system, ' slit, rectype: rectype-eval
' no.extensions dup dup rectype: rectype-null
' noop ' lit, dup rectype: rectype-num
' noop ' 2lit, dup rectype: rectype-dnum

:noname name?int execute-;s ;
' name-compsem
' lit,
rectype: rectype-name ( takes nt, i.e. result of find-name and find-name-in )

:noname drop execute ;
:noname 0> IF execute ELSE compile, THEN ;
' 2lit,
rectype: rectype-word ( takes xt +/-1, i.e. result of find and search-wordlist )

:noname ( object xt -- ) swap >o execute o> ; ' >oo> ' lit,
rectype: rectype-moof2

' interpret-notfound1 ' compiler-notfound1 ' postpone-notfound1
rectype: rectype-notfound

Consider how these would change for an Option 2 system.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

[Stephen Pelc]

On Sun, 28 Jun 2020 17:14:50 GMT, an...@mips.complang.tuwien.ac.at
(Anton Ertl) wrote:

>Alex McDonald <al...@rivadpm.com> writes:
>>2. Are there to be further discussions on the 2 cell auto-magic POSTPONE
>>action, or does Stephen Pelc's comment "We cannot predict how
>>recognisers will be used, so attempting to automate the POSTPONE actions
>>is doomed to failure" stand?

I stand by what I said as quoted above. It way well be possible to
automate the POSTPONE action of literals, but recognisers have been
promoted and implemented for jobs other than literals, for example
OOP packages. We (MPE) have converted two oop packages (ClassVFX
and CIAO) to use recognisers.

Because we converted the packages rather than completely rewrote
the parsing portions, they are not pretty and the normal three-
action structure is only part of the compilation and interpretation
system. I can see a way forward to a cleaner implementation, but
there is not enough use and demand yet.

As to POSTPONE for elements such as (Class).x I haven't got a clue
yet. In addition, my experience with Forth gurus is that as soon as
they have a need, they will twist the system as required and fix the
standards breakages later. Do not underestimate what a good Forth
programmer can do, especially thos with tag lines about "doing it
yourself".

Stephen


--
Stephen Pelc, ste...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612
web: http://www.mpeforth.com - free VFX Forth downloads

[Ruvim]

On 2020-06-28 21:12, Stephen Pelc wrote:
> On Sun, 28 Jun 2020 17:14:50 GMT, an...@mips.complang.tuwien.ac.at
> (Anton Ertl) wrote:
>
>> Alex McDonald <al...@rivadpm.com> writes:
>>> 2. Are there to be further discussions on the 2 cell auto-magic POSTPONE
>>> action, or does Stephen Pelc's comment "We cannot predict how
>>> recognisers will be used, so attempting to automate the POSTPONE actions
>>> is doomed to failure" stand?
>
> I stand by what I said as quoted above.

Could you please provide a particular example of the mentioned failure?

> It way well be possible to automate the POSTPONE action of literals,

What basic factors do you mean? I.e., to automate via what?

> but recognisers have been
> promoted and implemented for jobs other than literals, for example
> OOP packages. We (MPE) have converted two oop packages (ClassVFX
> and CIAO) to use recognisers.
>
> Because we converted the packages rather than completely rewrote
> the parsing portions, they are not pretty and the normal three-
> action structure is only part of the compilation and interpretation
> system. I can see a way forward to a cleaner implementation, but
> there is not enough use and demand yet.

Do you mean that a user cannot *replace* the system's recognizer by his
own recognizer?

> As to POSTPONE for elements such as (Class).x I haven't got a clue
> yet.

If the phrase "(Class).x" does not have an immediate argument (i.e. its
compilation semantics don't involve parsing the input buffer),
then the phrase

POSTPONE (Class).x

is equivalent to

[: (Class).x ;] COMPILE,


or, if it is mapped to an ordinary Forth definition, to

['] (Class).x COMPILE,



--
Ruvim

[Stephen Pelc]

On Mon, 29 Jun 2020 10:32:01 +0300, Ruvim <ruvim...@gmail.com>
wrote:

>On 2020-06-28 21:12, Stephen Pelc wrote:
>> On Sun, 28 Jun 2020 17:14:50 GMT, an...@mips.complang.tuwien.ac.at
>> (Anton Ertl) wrote:
>>
>>> Alex McDonald <al...@rivadpm.com> writes:
>>>> 2. Are there to be further discussions on the 2 cell auto-magic POSTPONE
>>>> action, or does Stephen Pelc's comment "We cannot predict how
>>>> recognisers will be used, so attempting to automate the POSTPONE actions
>>>> is doomed to failure" stand?
>>
>> I stand by what I said as quoted above.
>
>Could you please provide a particular example of the mentioned failure?

(Class).x - neither (Class) nor x are literals.

>> It way well be possible to automate the POSTPONE action of literals,
>
>What basic factors do you mean? I.e., to automate via what?

See Anton's statements about generating the POSTPONE action. I'm
sure he's right about literals, but I'm sure he's wrong about all
recognisers being for literals.

>Do you mean that a user cannot *replace* the system's recognizer by his
>own recognizer?

No.

>If the phrase "(Class).x" does not have an immediate argument (i.e. its
>compilation semantics don't involve parsing the input buffer),
>then the phrase
>
> POSTPONE (Class).x
>
>is equivalent to
>
> [: (Class).x ;] COMPILE,
>
>
>or, if it is mapped to an ordinary Forth definition, to
>
> ['] (Class).x COMPILE,

That's a circular argument because you need the recogniser to
generate the equivalents.

Yes, you can probably do this stuff with nested definitions, but
that something is technically possible does not mean that we should
adopt it. In some cases, the best solution is for a recogniser to
declare that it cannot POSTPONE the item.

[peter....@gmail.com]

The problem are not the rectypes. It is postpone.
From a snapshot in January 2020 I see kernel/recognizer.fs

: >postpone ( ... rectype -- )
>postpone-replacer dup >r rectype>post execute r> rectype>comp compile, ;

: postpone ( "name" -- ) \ core
\g Compiles the compilation semantics of @i{name}.
parse-name forth-recognizer recognize >postpone
; immediate restrict

The problem is the compile, in >postpone. This is required by your option 2
system. In an option 1 system you would have just

>postpone-replacer dup >r rectype>post execute

The compile would be inside the rectype>post action.
When you use compile, as in your option 2 it is fixed to be the
system compile,
With option 1 I could use the recognizers for a meta or cross compiler.
The compile, needed in that case would be for the target system address
space and CPU architecture. That can easily be supplied with the right
rectype actions.

I think we have had this discussion before. I have implemented an
option 1 system and will continue with that. Also the rectype>post
action will be much easier to explain as it is complete and not
depending on a compile, hidden in postpone!

BR
Peter Fälth

[Ruvim]

On 2020-06-29 18:16, peter....@gmail.com wrote:
> On Sunday, 28 June 2020 19:49:59 UTC+2, Anton Ertl wrote:

[...]

>> Consider how these would change for an Option 2 system.
>
> The problem are not the rectypes. It is postpone.
> From a snapshot in January 2020 I see kernel/recognizer.fs
>
> : >postpone ( ... rectype -- )
> >postpone-replacer dup >r rectype>post execute r> rectype>comp compile, ;

(1)

> : postpone ( "name" -- ) \ core
> \g Compiles the compilation semantics of @i{name}.
> parse-name forth-recognizer recognize >postpone
> ; immediate restrict
>
> The problem is the compile, in >postpone. This is required by your option 2
> system. In an option 1 system you would have just
>
> >postpone-replacer dup >r rectype>post execute
>
> The compile would be inside the rectype>post action.
> When you use compile, as in your option 2 it is fixed to be the
> system compile,
> With option 1 I could use the recognizers for a meta or cross compiler.
> The compile, needed in that case would be for the target system address
> space and CPU architecture. That can easily be supplied with the right
> rectype actions.

Let's to consider an example:

: foo postpone bar ;

If this foo is created in the host Forth system, the "compile," word in
the line (1) above should be from the host system. When foo is executed,
it can generate code for the target system according to the result of
recognizing "bar".

If this foo is created in the target Forth system, the special
"postpone" word should be used. As well as in the case of any other
code-generating word like "if", ">r", "exit", etc.

It doesn't make much sense to use "postpone" from the host system, if
you have to use special versions for all other code-generating words.


Therefore, nothing prevents us to use recognizers for cross compiling in
either implementation.


--
Ruvim

[peter....@gmail.com]

Well of course it does not prevent us! Meta and cross compiling have existed
from the beginning of Forth. The same well proven techniques can be used
also when the system uses recognizers. What I am trying to explore is using recognizers to simplify meta/cross compiling.

I am also looking at having a postpone that just sets a state and the
postponing is then handles in the interpreter loop. One postpone state
for just postponing the next word and one for postponing until some
super-immediate word. These are just ideas for the moment

BR
Peter


> --
> Ruvim

[Ruvim]

Recognizer API allows to reuse the Forth text interpreter of the host
system to compile sources into the form (and the dictionary) of the
target system. It means that INCLUDED and EVALUATE from the host system
also work.

But it really helps if otherwise you want to implement a special Forth
text interpreter for the target compilation (and also re-implement
EVALUATE and INCLUDED).

Some other approaches don't require to implement the special Forth text
interpreter. There are:

a. upfront source-to-source translation (when a suitable markup of the
source codes is used);

b. compile the target system by means of the target system translator
that is loaded into the host system (comprehensive modularization is
required).

> I am also looking at having a postpone that just sets a state and the
> postponing is then handles in the interpreter loop. One postpone state
> for just postponing the next word and one for postponing until some
> super-immediate word. These are just ideas for the moment

It works. See "]] ... [[" construct in Gforth

And more advanced "c{ ...}c" construct -- https://git.io/JJfi8


--
Ruvim

[Anton Ertl]

peter....@gmail.com writes:
>> So yes, you need three actions. The only question is whether
>> postponing is performed by 1) time-shifting the data followed by
>> time-shifting the compilation action (RFC-D), or whether 2) you
>> combine these in one action (IIRC RFC-C). Option 1 is a little harder
>> to understand, but requires less code for many rectypes, and I think
>> it is also less error-prone because of that.

...

>The problem are not the rectypes. It is postpone.
>From a snapshot in January 2020 I see kernel/recognizer.fs
>
>: >postpone ( ... rectype -- )

> >postpone-replacer dup >r rectype>post execute r> rectype>comp compile,=
> ;=20

>
>: postpone ( "name" -- ) \ core
> \g Compiles the compilation semantics of @i{name}.
> parse-name forth-recognizer recognize >postpone

>; immediate restrict=20

>
>The problem is the compile, in >postpone. This is required by your option 2
>system. In an option 1 system you would have just
>

> >postpone-replacer dup >r rectype>post execute=20

>
>The compile would be inside the rectype>post action.

>When you use compile, as in your option 2 it is fixed to be the=20

>system compile,
>With option 1 I could use the recognizers for a meta or cross compiler.

Yes, but to achieve that, you have to create cross-compiler versions
of all recognizers. If you load a third-party program that includes a
recognizer, you would have to change the recognizer code into a
cross-compiler version.

Overall, it seems to me that recognizers are the wrong place for
dealing with cross-compilers. I am not an expert on cross-compilers,
but it seems to me that having a jump vector table that contains
COMPILE, LITERAL IF etc., and being able to point the vector table
base to the cross-compiler versions is a better approach.

>I think we have had this discussion before.

Possibly. Thanks for reminding me.

[Ruvim]

On 2020-06-28 20:14, Anton Ertl wrote:
[...]

It is a weak argument since postponing action can be easy generated from
compilation action and reproducing action (aka "time-shifting").

: generate-postpone ( xt-compile xt-reproduce -- xt-postpone )
2>r :noname 2r> compile, lit, ['] compile, compile, postpone ;
;

: rectype:
over swap generate-postpone rectype:
;


An API that accepts postponing action just provides more options.

1. Actually, it can accept reproducing as well as postponing action.

2. It allows to create a better optimized code.

e.g. [: slit, '2lit, xt, ;] instead of [: slit, 'slit, xt, ;]
for string literals.

3. It allows to move additional parsing from a recognizer to each of
these three actions (but then postponing action does not reflect
semantics of POSTPONE anymore).

I.e., it allows to provide a *non-standard* postponing action



--
Ruvim

[Ruvim]

On 2020-07-02 20:15, Anton Ertl wrote:
> peter....@gmail.com writes:
[...]

>> With option 1 I could use the recognizers for a meta or cross compiler.
>
> Yes, but to achieve that, you have to create cross-compiler versions
> of all recognizers. If you load a third-party program that includes a
> If you load a third-party program that includes a
> recognizer, you would have to change the recognizer code into a
> cross-compiler version.
>
> Overall, it seems to me that recognizers are the wrong place for
> dealing with cross-compilers. I am not an expert on cross-compilers,
> but it seems to me that having a jump vector table that contains
> COMPILE, LITERAL IF etc., and being able to point the vector table
> base to the cross-compiler versions is a better approach.

This table doesn't solve the problem that can be solved via recognizers.

Namely, the problem is: during compilation for the target system, how to
find the words in the target dictionary? In other words, how to resolve
the names, the definitions for which was created in the target image?

A possible way is to use a recognizer that properly resolves the names
in the target dictionary.


--
Ruvim

[a...@littlepinkcloud.invalid]

You look them up in a wordlist dedicated to the purpose. Recognizers
aren't useful.

Andrew.

[Ruvim]

In the general case, this wordlist is in the different format and its id
is not suitable for FIND of the host system. I.e., this wordlist is
crated not by WORDLIST of the host system. So, it cannot be appended
into the search order of the host system.

Recognizers are useful in this case. Of course, another approaches can
be used too.


--
Ruvim

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
> On 2020-07-05 11:43, a...@littlepinkcloud.invalid wrote:
>> Ruvim <ruvim...@gmail.com> wrote:
>>> On 2020-07-02 20:15, Anton Ertl wrote:
>>>>
>>>> Overall, it seems to me that recognizers are the wrong place for
>>>> dealing with cross-compilers. I am not an expert on cross-compilers,
>>>> but it seems to me that having a jump vector table that contains
>>>> COMPILE, LITERAL IF etc., and being able to point the vector table
>>>> base to the cross-compiler versions is a better approach.
>>>
>>> This table doesn't solve the problem that can be solved via recognizers.
>>>
>>> Namely, the problem is: during compilation for the target system, how to
>>> find the words in the target dictionary? In other words, how to resolve
>>> the names, the definitions for which was created in the target image?
>>
>> You look them up in a wordlist dedicated to the purpose. Recognizers
>> aren't useful.
>
> In the general case, this wordlist is in the different format and its id
> is not suitable for FIND of the host system.

Why? Of course you have to lay out the dictionary on the target
system, but you can also have a wordlist on the host in which each
entry in this wordlist is a constant that points to the address on the
target. This is how it was done decades ago, and it still works
today. Why complicate things?

Andrew.

[Ruvim]

I can guess, it was done decades ago in some system dependent way. But
the Recognizer API (when it is standardized) provides a portable way of
reuse the Forth text interpreter of the host system.

Yes, we can have a wordlist on the host with the shadow definitions
(that are created in parallel). But each entry is not an arbitrary
address that we want, but a definition in the host system and the
corresponding xt in the host system.

The Forth text interpreter of the host system can get xt of a shadow
definition, but how it can compiler it into the target dictionary? It
cannot. The only possible way is to make these shadow definitions
immediate, so they will compile the corresponding target definitions by
themselves.

If we create a new word list in the target system, we should also create
the new corresponding shadow word list in the host system, and we should
change the search order and the compilation word list in parallel too.


It seems that use of the appropriate recognizer is a far simpler
solution. And in any case, one cannot say that recognizers are not
useful for cross compilation. They can take place in some approaches.


--
Ruvim

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
>>
>> Why? Of course you have to lay out the dictionary on the target
>> system, but you can also have a wordlist on the host in which each
>> entry in this wordlist is a constant that points to the address on the
>> target. This is how it was done decades ago, and it still works
>> today. Why complicate things?
>
> I can guess, it was done decades ago in some system dependent way.

Some of it was system dependent, but not this part.

> But the Recognizer API (when it is standardized) provides a portable
> way of reuse the Forth text interpreter of the host system.
>
> Yes, we can have a wordlist on the host with the shadow definitions
> (that are created in parallel). But each entry is not an arbitrary
> address that we want, but a definition in the host system and the
> corresponding xt in the host system.
>
> The Forth text interpreter of the host system can get xt of a shadow
> definition, but how it can compiler it into the target dictionary?

There is no need to use the text interpreter of the host system. Doing
so complicates the whole business: a better factoring (IMO, YMMV, &c.)
has two text interpreters, one for host words and one for target
words. It's better to separate words than to use a single word with
vectoring in the middle.

The text interpreter itself is far simpler than the cross compiler as
a whole, and so I fail to see the motivation to use the interpreter on
the host. Using recognizers instead of just writing another
interpreter loop is pointless.

Andrew.

[Ruvim]

On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
> Ruvim <ruvim...@gmail.com> wrote:
>>>
>>> Why? Of course you have to lay out the dictionary on the target
>>> system, but you can also have a wordlist on the host in which each
>>> entry in this wordlist is a constant that points to the address on the
>>> target. This is how it was done decades ago, and it still works
>>> today. Why complicate things?
>>
>> I can guess, it was done decades ago in some system dependent way.
>
> Some of it was system dependent, but not this part.
>
>> But the Recognizer API (when it is standardized) provides a portable
>> way of reuse the Forth text interpreter of the host system.
>>
>> Yes, we can have a wordlist on the host with the shadow definitions
>> (that are created in parallel). But each entry is not an arbitrary
>> address that we want, but a definition in the host system and the
>> corresponding xt in the host system.
>>
>> The Forth text interpreter of the host system can get xt of a shadow
>> definition, but how it can compiler it into the target dictionary?
>
> There is no need to use the text interpreter of the host system.

Well, if you don't use this text interpreter, why do you want to use a
wordlist of the host system as a mapping to the target wordlist?

> Doing
> so complicates the whole business: a better factoring (IMO, YMMV, &c.)
> has two text interpreters, one for host words and one for target
> words. It's better to separate words than to use a single word with
> vectoring in the middle.

It seems you missed my point in the first message by this subject
(see news:rdeq20$a1m$1...@dont-email.me). You compare the different
approaches (i.e, one interpreter vs two interpreters), but I compare the
same approach having the recognizers and without them.

There are many approaches in Forth cross compiling. I don't compare
them, what is better or isn't. I only pointed out that *some* of them
can benefit from using recognizers.

Particularly, recognizers can help if otherwise we want to implement a
special Forth text interpreter for the target compilation and also
re-implement EVALUATE and INCLUDED.

Perhaps, the approach I have in my mind is different from yours.
By my experience, the text interpreter that generates code for the
target system should resolve names in the host dictionary as well as in
the target dictionary. Namely, during cross compilation, the immediate
words should be taken from the special cross-compiling wordlist in the
host system, all other ordinary words — from the target dictionary.

> The text interpreter itself is far simpler than the cross compiler as
> a whole, and so I fail to see the motivation to use the interpreter on
> the host. Using recognizers instead of just writing another
> interpreter loop is pointless.

Yes, the text interpreter itself is a very simple thing.
But if you want INCLUDE or EVALUATE uses the new interpreter, you have
to re-implement them and also INCLUDED, REFILL, SOURCE, PARSE,
PARSE-NAME, and all the words that use them, e.g. "\", "(", "[IF]", etc.

So it is not only another interpreter loop, but also the full
corresponding layer of a Forth system. Now it should not look so
pointless to use the recognizers.


--
Ruvim

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
>> Ruvim <ruvim...@gmail.com> wrote:
>>>>
>>>> Why? Of course you have to lay out the dictionary on the target
>>>> system, but you can also have a wordlist on the host in which each
>>>> entry in this wordlist is a constant that points to the address on the
>>>> target. This is how it was done decades ago, and it still works
>>>> today. Why complicate things?
>>>
>>> I can guess, it was done decades ago in some system dependent way.
>>
>> Some of it was system dependent, but not this part.
>>
>>> But the Recognizer API (when it is standardized) provides a portable
>>> way of reuse the Forth text interpreter of the host system.
>>>
>>> Yes, we can have a wordlist on the host with the shadow definitions
>>> (that are created in parallel). But each entry is not an arbitrary
>>> address that we want, but a definition in the host system and the
>>> corresponding xt in the host system.
>>>
>>> The Forth text interpreter of the host system can get xt of a shadow
>>> definition, but how it can compiler it into the target dictionary?
>>
>> There is no need to use the text interpreter of the host system.
>
> Well, if you don't use this text interpreter, why do you want to use a
> wordlist of the host system as a mapping to the target wordlist?

Any cross compiler needs some sort of data strcuture to map a name to
an address in the target system. This is Forth, so people often use
the host's dictionary to do that.

>> Doing so complicates the whole business: a better factoring (IMO,
>> YMMV, &c.) has two text interpreters, one for host words and one
>> for target words. It's better to separate words than to use a
>> single word with vectoring in the middle.

> It seems you missed my point in the first message by this subject
> (see news:rdeq20$a1m$1...@dont-email.me). You compare the different
> approaches (i.e, one interpreter vs two interpreters), but I compare the
> same approach having the recognizers and without them.
>
> There are many approaches in Forth cross compiling. I don't compare
> them, what is better or isn't. I only pointed out that *some* of them
> can benefit from using recognizers.

You do, but that's (IMO, YMMV) comparing the merits of two different
ways of making a mistake. In that context it's not reasonable to point
out that neither should be done.

> Perhaps, the approach I have in my mind is different from yours.
> By my experience, the text interpreter that generates code for the
> target system should resolve names in the host dictionary as well as in
> the target dictionary. Namely, during cross compilation, the immediate
> words should be taken from the special cross-compiling wordlist in the
> host system, all other ordinary words — from the target dictionary.

Certainly, yes. But the wordlist which contains the address of all of
the words in the target system can also contain immediate words which
run on the host.

>> The text interpreter itself is far simpler than the cross compiler as
>> a whole, and so I fail to see the motivation to use the interpreter on
>> the host. Using recognizers instead of just writing another
>> interpreter loop is pointless.
>
> Yes, the text interpreter itself is a very simple thing.
> But if you want INCLUDE or EVALUATE uses the new interpreter, you have
> to re-implement them and also INCLUDED, REFILL, SOURCE, PARSE,
> PARSE-NAME, and all the words that use them, e.g. "\", "(", "[IF]", etc.

Not at all. Parsing doesn't change. The text interpreter calls PARSE,
not the other way around.

It's necessary only to switch to the new interpreter when generating
code for the target. Everything else uses the host's interpreter.
There's no need, and indeed no point, using the new interpreter loop
for anything except when generating code for the target. To do so
makes things pointlessly difficult, IMO.

Andrew.

[Ruvim]

On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
> Ruvim <ruvim...@gmail.com> wrote:
>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:

[...]

>>> There is no need to use the text interpreter of the host system.
>>
>> Well, if you don't use this text interpreter, why do you want to use a
>> wordlist of the host system as a mapping to the target wordlist?
>
> Any cross compiler needs some sort of data strcuture to map a name to
> an address in the target system. This is Forth, so people often use
> the host's dictionary to do that.

OK, I see, it is certainly possible.
I just want to point out that the target word lists are also a ready
data structure that maps a name to and the address in the target system.

[...]

>>> The text interpreter itself is far simpler than the cross compiler as
>>> a whole, and so I fail to see the motivation to use the interpreter on
>>> the host. Using recognizers instead of just writing another
>>> interpreter loop is pointless.
>>
>> Yes, the text interpreter itself is a very simple thing.
>> But if you want INCLUDE or EVALUATE uses the new interpreter, you have
>> to re-implement them and also INCLUDED, REFILL, SOURCE, PARSE,
>> PARSE-NAME, and all the words that use them, e.g. "\", "(", "[IF]", etc.
>
> Not at all. Parsing doesn't change. The text interpreter calls PARSE,
> not the other way around.
>
> It's necessary only to switch to the new interpreter when generating
> code for the target. Everything else uses the host's interpreter.

How can you switch to the new interpreter and back?

Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
do parsing? I.e., they call the new interpreter that returns control
after ";" or ";]" ?

It is a possible approach, but it makes these words to have non standard
behavior.

I don't see how to stay in the standard basis and avoid re-implementing
of INCLUDE and Co., if we use the special text interpreter for
generating target code.


--
Ruvim

[none albert]

I agree. Suppose we have a straightforward meta wordlist
that can transform the lexeme's in a body of code into correct
meta tokens, then compile them according to the target machine.
This is accomplished by having meta on top of the search order.
Code like
: test TASK NOOP DO I . LOOP ;
should be a walk in the park, despite having immediate words.

Now suppose we have
: aap 10 0 DO SPACE LOOP ;
It certainly helps to have a PREFIX (recognizer) for integers in the
meta namespace that do a (NUMBER) on 10 0 than a LITERAL
both in a meta context.

Where recognizers are supposedly more powerful than my PREFIX'es
I guess they are usefull too, but they are not as easily skoped
into a meta namespace/VOCABULARY than PREFIX's are

I have the original document
METAFORTH (tm)
A metacompiler for FIG-FORTH

for meta compilation here, inherited from late fellow-forther Arie
Kattenberg who introduced metacompilation in the Netherlands.
The title should have been:
METAFORTH (tm)
A crosscompiler for FIG-FORTH

It is copyrighted John Cassady, maybe a job for Pintaske or Peter
Forth to make it available to the world, for it is certainly
an important historic document.
It is one of the better kept secrets, and you are in for a surprise:
from the 76 page document only a scant 3 and a quarter page and a diagram
is explanation, and later one page and a diagram.
You are supposed to study the 8080 and Forth code to learn
the cross compiler. And the hex dump's can be used to find
typo's.
The rareness and the (amateurish) coyright probably has caused the
legend status of the term metacompilation.
"All rights reserved. For a license for the commercial use of the
information contained herein ..."

Few companies even nowadays publish documents and then claim it is
illegal to use its information in any way, shape or form.

>--
>Ruvim

Groetjes Albert
--
This is the first day of the end of your life.
It may not kill you, but it does make your weaker.
If you can't beat them, too bad.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Andy Valencia]

a...@littlepinkcloud.invalid writes:
> Any cross compiler needs some sort of data strcuture to map a name to
> an address in the target system. This is Forth, so people often use
> the host's dictionary to do that.

Wow, bringing back a lot of memories when I was designing my own
metacompilation, and aph's comments certainly match my own experience.
Open source, if you care to take a peek:

https://vsta.org/cgi-bin/fview.py?path=/kernel/base.txt

Andy Valencia
Home page: https://www.vsta.org/andy/
To contact me: https://www.vsta.org/contact/andy.html

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
> On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
>> Ruvim <ruvim...@gmail.com> wrote:
>>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
> [...]
>>>> There is no need to use the text interpreter of the host system.
>>>
>>> Well, if you don't use this text interpreter, why do you want to use a
>>> wordlist of the host system as a mapping to the target wordlist?
>>
>> Any cross compiler needs some sort of data strcuture to map a name to
>> an address in the target system. This is Forth, so people often use
>> the host's dictionary to do that.
>
> OK, I see, it is certainly possible.
> I just want to point out that the target word lists are also a ready
> data structure that maps a name to and the address in the target system.

True, but if the target system is headless -- normal in an embedded
system -- there will be no dictionary in that system. Also, it's not
that unusual to make some of the words in the target statem headless.

>> It's necessary only to switch to the new interpreter when generating
>> code for the target. Everything else uses the host's interpreter.
>
> How can you switch to the new interpreter and back?
>
> Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
> do parsing? I.e., they call the new interpreter that returns control
> after ";" or ";]" ?

Yes, you can do that, or perhaps just have a word TARGET that starts
the new test interpreter, and put it at the top of an included
file. It's certainly very convenient IME to have any error while
cross-compiling reset to a state where you have a normal Forth system.

> It is a possible approach, but it makes these words to have non
> standard behavior.

Good! :-)

> I don't see how to stay in the standard basis and avoid
> re-implementing of INCLUDE and Co., if we use the special text
> interpreter for generating target code.

You make a good point, but given all of the odd things that a cross
compiler has to do, I can't see any way to stay standard anyway. In
particular, you still need to be able to tell the system which system
the does part of a CREATE ... DOES> word will execute on. If you're
defining things like CONSTANTs you might well need to generate two of
the runtime parts.

IMO, the job of cross compilation is so wildly different from that of
host compilation that it's not worth trying to hide the differences.

Andrew.

[Ruvim]

On 2020-07-06 19:35, a...@littlepinkcloud.invalid wrote:
> Ruvim <ruvim...@gmail.com> wrote:
>> On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
>>> Ruvim <ruvim...@gmail.com> wrote:
>>>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
>> [...]
>>>>> There is no need to use the text interpreter of the host system.
>>>>
>>>> Well, if you don't use this text interpreter, why do you want to use a
>>>> wordlist of the host system as a mapping to the target wordlist?
>>>
>>> Any cross compiler needs some sort of data strcuture to map a name to
>>> an address in the target system. This is Forth, so people often use
>>> the host's dictionary to do that.
>>
>> OK, I see, it is certainly possible.
>> I just want to point out that the target word lists are also a ready
>> data structure that maps a name to and the address in the target system.
>
> True, but if the target system is headless -- normal in an embedded
> system -- there will be no dictionary in that system. Also, it's not
> that unusual to make some of the words in the target statem headless.

Yes, in the case of headless target system we need a temporary mapping
(that will not be a part of the target image).

>>> It's necessary only to switch to the new interpreter when generating
>>> code for the target. Everything else uses the host's interpreter.
>>
>> How can you switch to the new interpreter and back?
>>
>> Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
>> do parsing? I.e., they call the new interpreter that returns control
>> after ";" or ";]" ?
>
> Yes, you can do that, or perhaps just have a word TARGET that starts
> the new test interpreter, and put it at the top of an included
> file.

So, Recognizer API is just one more method to switch to the new text
interpreter. This method is not worse and even has some advantages over
two other methods above.

When you set another recognizer for the system's text interpreter, you
actually switch to the new text interpreter. A feature is that INCLUDED
uses this new interpreter automatically. Also, you need less code to
define this interpreter.


Regarding a method of the special word at the top of an included file. I
use this method sometimes (example: https://git.io/JJLpO). In the
general case, this method requires to change the source files and make
them non standard. I mean the following scenario.

Let I have a module of the target system as several files in the
standard Forth (but with certain restrictions) — one main file and other
files that are included via INCLUDE. A requirement is to compile this
module into the target image without changing the source codes of this
module.

> It's certainly very convenient IME to have any error while
> cross-compiling reset to a state where you have a normal Forth system.

It is not a problem when we make nesting, i.e. change a state,
call/catch something and then restore the state.

When we switch the text interpreter via Recognizer API, we can do it
conceptually as the following:

: included-target ( c-addr u -- )
perceptor >r 'recognize-target set-perceptor
'included catch
r> set-perceptor throw
;

>> It is a possible approach, but it makes these words to have non
>> standard behavior.
>
> Good! :-)
>
>> I don't see how to stay in the standard basis and avoid
>> re-implementing of INCLUDE and Co., if we use the special text
>> interpreter for generating target code.
>
> You make a good point, but given all of the odd things that a cross
> compiler has to do, I can't see any way to stay standard anyway. In
> particular, you still need to be able to tell the system which system
> the does part of a CREATE ... DOES> word will execute on. If you're
> defining things like CONSTANTs you might well need to generate two of
> the runtime parts.

It's a subject of the restrictions on programs of the cross compiler.

In the simplest variant it can forbid to use in interpretation state or
to execute any words except the words from the provided list.

A more advanced cross compiler has less restrictions on programs.

> IMO, the job of cross compilation is so wildly different from that of
> host compilation that it's not worth trying to hide the differences.

I think a promising way is to minimize these differences and make them
very compact.


--
Ruvim

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
> On 2020-07-06 19:35, a...@littlepinkcloud.invalid wrote:
>> Ruvim <ruvim...@gmail.com> wrote:
>>> On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
>>>> Ruvim <ruvim...@gmail.com> wrote:
>>>>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
>>> [...]
>>>>>> There is no need to use the text interpreter of the host system.
>>

>>>> It's necessary only to switch to the new interpreter when generating
>>>> code for the target. Everything else uses the host's interpreter.
>>>
>>> How can you switch to the new interpreter and back?
>>>
>>> Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
>>> do parsing? I.e., they call the new interpreter that returns control
>>> after ";" or ";]" ?
>>
>> Yes, you can do that, or perhaps just have a word TARGET that starts

>> the new [text] interpreter, and put it at the top of an included

>> file.
>
> So, Recognizer API is just one more method to switch to the new text
> interpreter.

Indeed. Another black mark against the recognizer API... :-)

>> IMO, the job of cross compilation is so wildly different from that of
>> host compilation that it's not worth trying to hide the differences.
>
> I think a promising way is to minimize these differences and make
> them very compact.

In practice every attempt to hide the differences must break down
somewhere, and perhaps more importantly it goes against the
grain. Forth is a radically transparent programming system, where
there aren't any tricks going on under the table.

That's not to say that cross compiler shouldn't be as clean as
possible, but when you have a CREATE DOES> word (or indeed any kind of
metaprogramming) the systems has no way to know what you mean unless
you tell it. Trying to fake a Forth cross compiler so that it looks
like a host system is a losing battle. IMO.

Andrew.

[Ruvim]

On 2020-07-07 11:23, a...@littlepinkcloud.invalid wrote:
> Ruvim <ruvim...@gmail.com> wrote:
>> On 2020-07-06 19:35, a...@littlepinkcloud.invalid wrote:
>>> Ruvim <ruvim...@gmail.com> wrote:
>>>> On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
>>>>> Ruvim <ruvim...@gmail.com> wrote:
>>>>>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
>>>> [...]
>>>>>>> There is no need to use the text interpreter of the host system.
>>>
>>>>> It's necessary only to switch to the new interpreter when generating
>>>>> code for the target. Everything else uses the host's interpreter.
>>>>
>>>> How can you switch to the new interpreter and back?
>>>>
>>>> Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
>>>> do parsing? I.e., they call the new interpreter that returns control
>>>> after ";" or ";]" ?
>>>
>>> Yes, you can do that, or perhaps just have a word TARGET that starts
>>> the new [text] interpreter, and put it at the top of an included
>>> file.
>>
>> So, Recognizer API is just one more method to switch to the new text
>> interpreter.
>
> Indeed. Another black mark against the recognizer API... :-)

What makes you think so?

>>> IMO, the job of cross compilation is so wildly different from that of
>>> host compilation that it's not worth trying to hide the differences.
>>
>> I think a promising way is to minimize these differences and make
>> them very compact.
>
> In practice every attempt to hide the differences must break down
> somewhere, and perhaps more importantly it goes against the
> grain. Forth is a radically transparent programming system, where
> there aren't any tricks going on under the table.
>
> That's not to say that cross compiler shouldn't be as clean as
> possible, but when you have a CREATE DOES> word (or indeed any kind of
> metaprogramming) the systems has no way to know what you mean unless
> you tell it.

Here we have two options.

a. It is an extension of the cross compiler. In this case you tell the
system what is what.

b. It is a part of the target system. I this case it should not know
anything about the cross compiler (as the best case).

> Trying to fake a Forth cross compiler so that it looks
> like a host system is a losing battle. IMO.

Not in general. But in some parts it is possible. If we just use a
string library (e.g. https://git.io/JJtv9), why it should know anything
about cross-compiler? It shouldn't.

But an interesting question is, can we make a cross compiler that allows
to transparently include into the target system and *use* a control flow
library? (e.g. https://git.io/JfiJB)


--
Ruvim

[Stephen Pelc]

On Tue, 07 Jul 2020 03:23:47 -0500, a...@littlepinkcloud.invalid wrote:

>That's not to say that cross compiler shouldn't be as clean as
>possible, but when you have a CREATE DOES> word (or indeed any kind of
>metaprogramming) the systems has no way to know what you mean unless
>you tell it.

The MPE cross compiler has handled the majority of defining words
"automagically" since 1982. Note that it's only the majority and
you would hate the implementation.

> Trying to fake a Forth cross compiler so that it looks
>like a host system is a losing battle. IMO.

A target for me with a cross compiler is code transparency, by which I
mean that the code for a cross compiled version should be as close as
possible to that for a "normal" hosted version.

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
> On 2020-07-07 11:23, a...@littlepinkcloud.invalid wrote:
>> Ruvim <ruvim...@gmail.com> wrote:
>>> On 2020-07-06 19:35, a...@littlepinkcloud.invalid wrote:
>>>> Ruvim <ruvim...@gmail.com> wrote:
>>>>> On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
>>>>>> Ruvim <ruvim...@gmail.com> wrote:
>>>>>>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
>>>>> [...]
>>>>>>>> There is no need to use the text interpreter of the host system.
>>>>
>>>>>> It's necessary only to switch to the new interpreter when generating
>>>>>> code for the target. Everything else uses the host's interpreter.
>>>>>
>>>>> How can you switch to the new interpreter and back?
>>>>>
>>>>> Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
>>>>> do parsing? I.e., they call the new interpreter that returns control
>>>>> after ";" or ";]" ?
>>>>
>>>> Yes, you can do that, or perhaps just have a word TARGET that starts
>>>> the new [text] interpreter, and put it at the top of an included
>>>> file.
>>>
>>> So, Recognizer API is just one more method to switch to the new text
>>> interpreter.
>>
>> Indeed. Another black mark against the recognizer API... :-)
>
> What makes you think so?

It's made me realize that recognizers migh be problematic when cross
compiling, so that's another thing to think about. I'm not at all sure
about that, though. I don't know if anyone has managed to cross
compile code that uses recognizers.

>> That's not to say that cross compiler shouldn't be as clean as
>> possible, but when you have a CREATE DOES> word (or indeed any kind of
>> metaprogramming) the systems has no way to know what you mean unless
>> you tell it.
>
> Here we have two options.
>
> a. It is an extension of the cross compiler. In this case you tell the
> system what is what.
>
> b. It is a part of the target system. I this case it should not know
> anything about the cross compiler (as the best case).

So something like

: bumps ( n) create , does> ( n - n') @ + ;

... later ...

1024 bumps +block
64 bumps +line

wouldn't work when cross compiling, because this newly-created word
CONSTANT wouldn't be executable on the host at compile time.

>> Trying to fake a Forth cross compiler so that it looks
>> like a host system is a losing battle. IMO.
>
> Not in general. But in some parts it is possible. If we just use a
> string library (e.g. https://git.io/JJtv9), why it should know anything
> about cross-compiler? It shouldn't.

See the simple example above. There needs to be some way to convey
the intent of the programmer.

> But an interesting question is, can we make a cross compiler that allows
> to transparently include into the target system and *use* a control flow
> library? (e.g. https://git.io/JfiJB)

The question I'd ask is: does the target system have in interpreter of
its own? And, if so, would you want the words ?E et al. to be
in the dictionary on the target system? As well as on the host
system?

Andrew.

[none albert]

In article <_4KdncOTfdaux5nC...@supernews.com>,

You can't just dump that on us in the context of cross compilation.

I'm currently working on Albert Nijhof's meta compiler for rsic V.
It is always

target:

: bumps ( n) create , does> ( n - n') @ + ;

or
meta:

: bumps ( n) create , does> ( n - n') @ + ;

Now which one is it?

VOCABULARY target
: target: target DEFINITIONS ;

>
>
>Andrew.

[a...@littlepinkcloud.invalid]

Stephen Pelc <ste...@mpeforth.com> wrote:
> On Tue, 07 Jul 2020 03:23:47 -0500, a...@littlepinkcloud.invalid wrote:
>
>>That's not to say that cross compiler shouldn't be as clean as
>>possible, but when you have a CREATE DOES> word (or indeed any kind of
>>metaprogramming) the systems has no way to know what you mean unless
>>you tell it.
>
> The MPE cross compiler has handled the majority of defining words
> "automagically" since 1982. Note that it's only the majority and
> you would hate the implementation.

Probably. But, out of curiosity, would something like

: constant create , does> @ ;

create definitions for CONSTANT in both the host and the target
dictionaries, and place the runtime DOES> part in both?

Andrew.

[Stephen Pelc]

On Tue, 07 Jul 2020 06:19:15 -0500, a...@littlepinkcloud.invalid wrote:

>It's made me realize that recognizers migh be problematic when cross
>compiling, so that's another thing to think about. I'm not at all sure
>about that, though. I don't know if anyone has managed to cross
>compile code that uses recognizers.

Every day for the last x months. Recognisers need no special support
from the cross compiler. However, some may be needed to compile the
new notations. In other words, a recogniser implementation needs
no cross compiler support. Recogniser use needs the three actions,
interpret, compile and postpone. For literal recognition, I see no
issues. For stuff like OOP recognisers, I'll defer comment until
we have done it.

[Ruvim]

On 2020-07-07 14:19, a...@littlepinkcloud.invalid wrote:
> Ruvim <ruvim...@gmail.com> wrote:
>> On 2020-07-07 11:23, a...@littlepinkcloud.invalid wrote:
>>> Ruvim <ruvim...@gmail.com> wrote:
>>>> On 2020-07-06 19:35, a...@littlepinkcloud.invalid wrote:
>>>>> Ruvim <ruvim...@gmail.com> wrote:
>>>>>> On 2020-07-06 11:31, a...@littlepinkcloud.invalid wrote:
>>>>>>> Ruvim <ruvim...@gmail.com> wrote:
>>>>>>>> On 2020-07-05 22:09, a...@littlepinkcloud.invalid wrote:
>>>>>> [...]
>>>>>>>>> There is no need to use the text interpreter of the host system.
>>>>>
>>>>>>> It's necessary only to switch to the new interpreter when generating
>>>>>>> code for the target. Everything else uses the host's interpreter.
>>>>>>
>>>>>> How can you switch to the new interpreter and back?
>>>>>>
>>>>>> Do you mean that ":", ":noname", "[:" from the cross-compiling wordlist
>>>>>> do parsing? I.e., they call the new interpreter that returns control
>>>>>> after ";" or ";]" ?
>>>>>
>>>>> Yes, you can do that, or perhaps just have a word TARGET that starts
>>>>> the new [text] interpreter, and put it at the top of an included
>>>>> file.
>>>>
>>>> So, Recognizer API is just one more method to switch to the new text
>>>> interpreter.
>>>
>>> Indeed.

Well, it least you have realized this :)

>>> Another black mark against the recognizer API... :-)
>>
>> What makes you think so?
>
> It's made me realize that recognizers migh be problematic when cross
> compiling, so that's another thing to think about. I'm not at all sure
> about that, though. I don't know if anyone has managed to cross
> compile code that uses recognizers.

This problem isn't more hard than CREATE DOES>

In the simplest case a cross compiler just does not provide the
Recognizer API for the programs (although, it can use this API of the
host system).

The next option is to separate extensions of the cross compiler and the
target program. I.e., to provide Recognizer API for extensions of the
cross compiler.

>>> That's not to say that cross compiler shouldn't be as clean as
>>> possible, but when you have a CREATE DOES> word (or indeed any kind of
>>> metaprogramming) the systems has no way to know what you mean unless
>>> you tell it.
>>
>> Here we have two options.
>>
>> a. It is an extension of the cross compiler. In this case you tell the
>> system what is what.
>>
>> b. It is a part of the target system. I this case it should not know
>> anything about the cross compiler (as the best case).
>
> So something like
>
> : bumps ( n) create , does> ( n - n') @ + ;
>

> .... later ...

>
> 1024 bumps +block
> 64 bumps +line
>
> wouldn't work when cross compiling, because this newly-created word
> CONSTANT wouldn't be executable on the host at compile time.

Yes, in the straight-forward approach it cannot be executed on the host.
Actually, in this case you cannot even use "bumps" interpretively, you
should get an error that "bumps" cannot be executed.

But this problem has solutions.

If the cross compiler supports extensions, you should define "bumps" as

an extension of the cross compiler.

In more advanced case the cross compiler generates two definitions in
parallel — a definition in the target dictionary, and the corresponding
shadow anonymous definition that can be executed on the host at compile
time.

So, when you try to execute a definition from the target dictionary, the
interpreter actually executes the shadow definition: it resolves the
target xt into the corresponding shadow xt and execute it. (1)


Yet another solution is emulation: the cross compiler executes the
definitions from the target dictionary by means of emulation.

>>> Trying to fake a Forth cross compiler so that it looks
>>> like a host system is a losing battle. IMO.
>>
>> Not in general. But in some parts it is possible. If we just use a
>> string library (e.g. https://git.io/JJtv9), why it should know anything
>> about cross-compiler? It shouldn't.
>
> See the simple example above. There needs to be some way to convey
> the intent of the programmer.
>
>> But an interesting question is, can we make a cross compiler that allows
>> to transparently include into the target system and *use* a control flow
>> library? (e.g. https://git.io/JfiJB)
>
> The question I'd ask is: does the target system have in interpreter of
> its own? And, if so, would you want the words ?E et al. to be
> in the dictionary on the target system? As well as on the host
> system?

If these control flow words are in the target dictionary, but they are
not used in the code — it works even in the simplest case.

If they should not be in the target dictionary, — they are just an

extension of the cross compiler.

The hardest case is that they should be in the target dictionary, and
they are used in the code.

So, the module can be included twice: as an extension of the cross
compiler and as a part of the target system.


The question is: can we make a cross compiler that allows to include
this module only once - as a part of the target system, and also it
allows to use these control flow words.

It seems, it is feasible by means of (1) above.


--
Ruvim

[a...@littlepinkcloud.invalid]

Ruvim <ruvim...@gmail.com> wrote:
> On 2020-07-07 14:19, a...@littlepinkcloud.invalid wrote:
>> Ruvim <ruvim...@gmail.com> wrote:
>>> On 2020-07-07 11:23, a...@littlepinkcloud.invalid wrote:
>>>> Ruvim <ruvim...@gmail.com> wrote:
>>> Here we have two options.
>>>
>>> a. It is an extension of the cross compiler. In this case you tell the
>>> system what is what.
>>>
>>> b. It is a part of the target system. I this case it should not know
>>> anything about the cross compiler (as the best case).
>>
>> So something like
>>
>> : bumps ( n) create , does> ( n - n') @ + ;
>>
>> .... later ...
>>
>> 1024 bumps +block
>> 64 bumps +line
>>
>> wouldn't work when cross compiling, because this newly-created word
>> CONSTANT wouldn't be executable on the host at compile time.
>
> Yes, in the straight-forward approach it cannot be executed on the host.
> Actually, in this case you cannot even use "bumps" interpretively, you
> should get an error that "bumps" cannot be executed.
>
> But this problem has solutions.

Of course, and people have been using them for decades. You tell the
system what you mean, by using either HOST : or TARGET : . (Or
equivalent; I'm not hung up on syntax.)

> The question is: can we make a cross compiler that allows to include
> this module only once - as a part of the target system, and also it
> allows to use these control flow words.
>
> It seems, it is feasible by means of (1) above.

Probably, yes, if and only if all the dependencies of compiler
extension words were to be compiled on both the host and the
target. But should it be done, and if it were done would it prove in
practice to be useful? That's the most important question, IMO.

Andrew.