Stream and SPADCALL and Lisp

[Ralf Hemmecke]

Dear Waldek,

In Stream, I see

-- This description of the rep is not quite true.
-- The Rep is a pair of one of three forms:
-- [value: S, rest: %]
-- [nullstream: Magic, NIL ]
-- [nonnullstream: Magic, fun: () -> %]
-- [uninitializedstream : Magic, 0]
-- Could use a record of unions if we could guarantee no tags.

NullStream : S := _$NullStream$Lisp pretend S
NonNullStream : S := _$NonNullStream$Lisp pretend S
UninitializedStream : S := _$UninitializedStream$Lisp pretend S

Rep := Record(firstElt : S, restOfStream : %)

https://github.com/fricas/fricas/blob/master/src/algebra/stream.spad#L626

And also

-- we have to use SPADCALL directly to avoid wrong code
lazyEval x == SPADCALL(rst(x))$Lisp pretend %

https://github.com/fricas/fricas/blob/master/src/algebra/stream.spad#L626

May I ask whether

lazyEval x == (rst(x) pretend () -> %)()

would actually also work? Or is it that what produces wrong code?

The idea in Stream's representation is anyway that values of type % and
values of type ()->% can be stored in the same record position.

I am asking, because that would eliminate an explicit SPADCALL$Lisp.

In fact, I'd be happy, if all the function that are Lisp-specific like
EQ$Lisp, APPEND$Lisp are defined at the beginning and then no further
...$Lisp will appear.

And as you said in another mail, obviously Stream is not as lazy as it
could be. Maybe a good task for a student who wants to program in SPAD.

Ralf

[Waldek Hebisch]

On Tue, May 31, 2022 at 09:44:12PM +0200, Ralf Hemmecke wrote:
> Dear Waldek,
>
> In Stream, I see
>
> -- This description of the rep is not quite true.
> -- The Rep is a pair of one of three forms:
> -- [value: S, rest: %]
> -- [nullstream: Magic, NIL ]
> -- [nonnullstream: Magic, fun: () -> %]
> -- [uninitializedstream : Magic, 0]
> -- Could use a record of unions if we could guarantee no tags.
>
> NullStream : S := _$NullStream$Lisp pretend S
> NonNullStream : S := _$NonNullStream$Lisp pretend S
> UninitializedStream : S := _$UninitializedStream$Lisp pretend S
>
> Rep := Record(firstElt : S, restOfStream : %)
>
> https://github.com/fricas/fricas/blob/master/src/algebra/stream.spad#L626
>
> And also
>
> -- we have to use SPADCALL directly to avoid wrong code
> lazyEval x == SPADCALL(rst(x))$Lisp pretend %
>
> https://github.com/fricas/fricas/blob/master/src/algebra/stream.spad#L626
>
> May I ask whether
>
> lazyEval x == (rst(x) pretend () -> %)()
>
> would actually also work? Or is it that what produces wrong code?

Well, we had

lazyEval x == (rst(x):(()-> %)) ()

':' was doing what you apparently think that 'pretend' would do.
':' is gone, IIRC 'pretend' failed, so that is why we have SPADCALL...

There is extra thing: now we generate Lisp type declarations based
on Spad types. If there is enough type information and one uses
'pretend' to cheat about types, then we may get wrong code from
Lisp compiler (like type error in safe mode, crash in unsafe mode).

> The idea in Stream's representation is anyway that values of type % and
> values of type ()->% can be stored in the same record position.
>
> I am asking, because that would eliminate an explicit SPADCALL$Lisp.

Well, this is one of darkest corners in our code.

> In fact, I'd be happy, if all the function that are Lisp-specific like
> EQ$Lisp, APPEND$Lisp are defined at the beginning and then no further
> ...$Lisp will appear.

Well, one possiblity is to have small number of "primitive"
domains, say "BuiltinInteger", "BuiltinSymbol", "BuiltinList".
The idea being that only "primitive" domains can directly use
Lisp operations. And if we provided non-Lisp implementation
of builtin domains then the rest of algebra would work
without Lisp. However, that is future.

I am affraid that 'lazyEval' would have to be part of
"BuiltinStream"...

--
Waldek Hebisch

[Ralf Hemmecke]

> Well, we had
>
> lazyEval x == (rst(x):(()-> %)) ()
>
> ':' was doing what you apparently think that 'pretend' would do.
> ':' is gone, IIRC 'pretend' failed, so that is why we have SPADCALL...

Oh, we had this? But honestly, I am happy that ':' is gone. When I read
the above line, I wouldn't understand it as I understand 'pretend'.
'pretend' for me is no operation, just telling the compiler that it show
override the type with what 'pretend' says.
Maybe internally it works differently, but the above is my layman's view.
BTW, I would have no problem if a 'pretend' crashes the system.
'pretend' is claiming that the user knows enough about the compiler to
make the claim of a potentially unsafe type cast.

Maybe like Lisp calls, 'pretend' should only appear in what you called
"primitive domains'.

Ralf

[Waldek Hebisch]

On Tue, May 31, 2022 at 10:52:47PM +0200, Ralf Hemmecke wrote:
> >Well, we had
> >
> > lazyEval x == (rst(x):(()-> %)) ()
> >
> >':' was doing what you apparently think that 'pretend' would do.
> >':' is gone, IIRC 'pretend' failed, so that is why we have SPADCALL...
>
> Oh, we had this? But honestly, I am happy that ':' is gone. When I read the
> above line, I wouldn't understand it as I understand 'pretend'.
> 'pretend' for me is no operation, just telling the compiler that it show
> override the type with what 'pretend' says.
> Maybe internally it works differently, but the above is my layman's view.

There are small, subtle effects which may change generated code.
Did you ever think why 'delay' with pile works? 'delay' expects
function as an argument, pile as argument produces a value.
Why this works when produced value is _not_ a function?

> BTW, I would have no problem if a 'pretend' crashes the system. 'pretend' is
> claiming that the user knows enough about the compiler to make the claim of
> a potentially unsafe type cast.
>
> Maybe like Lisp calls, 'pretend' should only appear in what you called
> "primitive domains'.

There are at least two cases where 'pretend' is safe without looking
at low level details:

- we can use 'pretend' to store anything in variable of type None.
We can retrive it later using 'pretend' to original type.

- there are cases when compiler does not know that code is type
correct. 'pretend' can be used to convince to accept the
code. This was use for implementation of some advanced
concepts in Aldor.

Coming back to representation of Stream, arguably correct representation
is

Rep := Record(firstElt : None, restOfStream : None)

Namely code stores various things in both fields, with None this
is OK. Not so using current types. OTOH that would require
bunch of 'pretends' in stream code (probably could be hidden
in a few macros).

--
Waldek Hebisch

[Ralf Hemmecke]

On 01.06.22 21:09, Waldek Hebisch wrote:
> Did you ever think why 'delay' with pile works? 'delay' expects
> function as an argument, pile as argument produces a value. Why this
> works when produced value is _not_ a function?

Oh, good that you come to it. Exactly, I would always have expected
something like

foo(...) == delay
() + ->
PILE

There must be some magic in the compiler to silently insert that missing
piece. Is there?

> Coming back to representation of Stream, arguably correct
> representation is
>
> Rep := Record(firstElt : None, restOfStream : None)
>
> Namely code stores various things in both fields, with None this is
> OK.

Although I do not quite like the name None, but switching to such a
representation would at least make it clear that some magic is going on.
I would even use

Record(firstElt : MagicOrValue, restOfStream : LazyStream)

with macros MagicOrValue and LazyStream that expand to None, but with
documentation about what they are placeholders for.

> OTOH that would require bunch of 'pretends' in stream code (probably
> could be hidden in a few macros).

No problem if there would be a number of more "primitive" macros of
functions that hide the 'pretend'.

But this is for future code improvement.

Ralf

[Waldek Hebisch]

On Wed, Jun 01, 2022 at 09:27:28PM +0200, Ralf Hemmecke wrote:
> On 01.06.22 21:09, Waldek Hebisch wrote:
> >Did you ever think why 'delay' with pile works? 'delay' expects function
> >as an argument, pile as argument produces a value. Why this
> >works when produced value is _not_ a function?
>
> Oh, good that you come to it. Exactly, I would always have expected
> something like
>
> foo(...) == delay
> () + ->
> PILE
>
> There must be some magic in the compiler to silently insert that missing
> piece. Is there?

Yes. When request type is function type compiler calls
compWithMappingMode which is doing needed magic. In fact,
compWithMappingMode can handle old form of anonymous functions
with '#1', '#2', etc., but those are rejected by the parser.

However, zero argument case still works and is used with 'delay'.

I supect that the same magic that works with 'delay' breaks
'pretend' in 'lazyEval'...

--
Waldek Hebisch

[Ralf Hemmecke]

> Yes. When request type is function type compiler calls
> compWithMappingMode which is doing needed magic. In fact,
> compWithMappingMode can handle old form of anonymous functions
> with '#1', '#2', etc., but those are rejected by the parser.
>
> However, zero argument case still works and is used with 'delay'.

If I am not wrong then it is also used for 0 and 1.

MagmaWithUnit() : Category == Magma with
--constants
1 : constant -> %

I always get confused by that.

In Aldor that would simply be

1: %

Aldor distinguishes between constants and nullary function of type
()->%. I think that makes sense, since a nullary function can stil have
side effects whereas a constant cannot.

Ralf

[Waldek Hebisch]

On Wed, Jun 01, 2022 at 10:24:29PM +0200, Ralf Hemmecke wrote:
> >Yes. When request type is function type compiler calls
> >compWithMappingMode which is doing needed magic. In fact,
> >compWithMappingMode can handle old form of anonymous functions
> >with '#1', '#2', etc., but those are rejected by the parser.
> >
> >However, zero argument case still works and is used with 'delay'.
>
> If I am not wrong then it is also used for 0 and 1.
>
> MagmaWithUnit() : Category == Magma with
> --constants
> 1 : constant -> %
>
> I always get confused by that.

I do not think so: here we have normal heading used by all mappings.

> In Aldor that would simply be
>
> 1: %
>
> Aldor distinguishes between constants and nullary function of type ()->%. I
> think that makes sense, since a nullary function can stil have side effects
> whereas a constant cannot.

What can have side effect and what can not is a design decision
for a language. Of course, people associate meaning to names like
'constant' so it is better to follow natural expectations.
But there is wide spectum of things which can be called constants
and various fine aspect of associated semantics.

--
Waldek Hebisch