weird behaviour

[Ralf Hemmecke]

Can someone expplain why the first line does not succeed?

Ralf

(1) -> p4 := entries partitions(4)

>> Error detected within library code:
infinite stream

(1) -> p := partitions(4)

(1) [[4], [3, 1], [2, 2], [2, 1, 1], [1, 1, 1, 1]]
Type:
Stream(List(Integer))
(2) -> p4 := entries p

(2) [[4], [3, 1], [2, 2], [2, 1, 1], [1, 1, 1, 1]]
Type:
List(List(Integer))

[Waldek Hebisch]

On Tue, Aug 23, 2022 at 12:51:24PM +0200, Ralf Hemmecke wrote:
> Can someone expplain why the first line does not succeed?
>
> Ralf
>
> (1) -> p4 := entries partitions(4)
>
> >> Error detected within library code:
> infinite stream

IIUC the official idiom always was:

p4 := entries(complete(partitions(4)))

Justification is that othewise one may trigger infinite loop
by accident. 'complete' is supposed to be used only to
force computation off all entries, so no risc of accident.

More generally, there are patterns of stream opeartions that
are "safe", that is there is no risc of infinite loop.
Unfortunately, safe patterns are limited in what they can
compute. So there is some tension about unsafe operations:
"power to people" folks want them all and available as
default, while others want to have mostly safe operations
with unsafe one viewed as exceptional and essentially
confined to a ghetto.

--
Waldek Hebisch

[Ralf Hemmecke]

> IIUC the official idiom always was:
>
> p4 := entries(complete(partitions(4)))

OK. Grummel.

> Justification is that othewise one may trigger infinite loop
> by accident. 'complete' is supposed to be used only to
> force computation off all entries, so no risc of accident.

I must admit that I do not see how it makes more sense to let

entries: % -> List S

return an infinite stream error and require to user to explicitly call
"complete" when the user knows that the stream is finite and "entries"
is supposed to return a list.

The only infinite case would be that the stream st contains a cycle, but
in this case complete(st) runs into an infinite loop. So we can actually
expect either that "entries" returns a list or runs indefinitely in case
the stream is not finite.

Note that the result of "entries" is not lazy any more. So I bet an
ordinary user expects that "entries" calls "complete" implicitly.

> with unsafe one viewed as exceptional and essentially
> confined to a ghetto.

I agree that restricting unsafe operations to a few functions makes
sense, but in this special case... I would rather declare "entries" to
be a unsafe function.

Introducing "computedEntries: % -> List(S)" makes probably no sense,
since nobody asked for it.

I can live with "(s = copy(s))@Boolean" returning false in general, but
letting "entries" abort the computation if it is not yet verified that
the stream is finite is another case.

So I propose that we include "complete" into the definition of
"entries". Or at least add a statement to the error message that one
should apply "complete" if one knows that the stream is finite.

The problem would not have appeared if partitions(n) would return a List
instead of a Stream, but admittedly for big n one might not actually
want all partitions at once so a lazy aggregate makes sense.

Maybe all this calls for an explicit type FiniteStream (or LazyList)
which is lazy, but otherwise behaves like List.

Ralf

[Waldek Hebisch]

On Tue, Aug 23, 2022 at 02:41:41PM +0200, Ralf Hemmecke wrote:
> > IIUC the official idiom always was:
> >
> > p4 := entries(complete(partitions(4)))
>
> OK. Grummel.
>
> > Justification is that othewise one may trigger infinite loop
> > by accident. 'complete' is supposed to be used only to
> > force computation off all entries, so no risc of accident.
>
> I must admit that I do not see how it makes more sense to let
>
> entries: % -> List S
>
> return an infinite stream error and require to user to explicitly call
> "complete" when the user knows that the stream is finite and "entries" is
> supposed to return a list.
>
> The only infinite case would be that the stream st contains a cycle, but in
> this case complete(st) runs into an infinite loop. So we can actually expect
> either that "entries" returns a list or runs indefinitely in case the stream
> is not finite.

Actually, "entries" can detect cyclic streams. So the only trouble
is due to potentially infinite streams. And "entries" refuses to
work with such streams.

> Note that the result of "entries" is not lazy any more. So I bet an ordinary
> user expects that "entries" calls "complete" implicitly.

Ordinary user expects miracles, so that is really not a criterion
what we should do.

> > with unsafe one viewed as exceptional and essentially
> > confined to a ghetto.
>
> I agree that restricting unsafe operations to a few functions makes sense,
> but in this special case... I would rather declare "entries" to be a unsafe
> function.
>
> Introducing "computedEntries: % -> List(S)" makes probably no sense, since
> nobody asked for it.
>
> I can live with "(s = copy(s))@Boolean" returning false in general, but
> letting "entries" abort the computation if it is not yet verified that the
> stream is finite is another case.
>
> So I propose that we include "complete" into the definition of "entries". Or
> at least add a statement to the error message that one should apply
> "complete" if one knows that the stream is finite.

Acutually, "entries" is just one of several functions. If you look
at LazyStreamAggregate you will see that a lot of functions return
"infinte stream" error. So LazyStreamAggregate is internally consistent,
if you want to work with all stream elements, call 'complete' first.

We could try to clarify error messages. One case is when the
discover cyclic stream, in such message could be

infinite (cyclic) stream

OTOH, in case of lazy stream we could have:

potentially infinite (lazy) stream, maybe use 'complete' first

> The problem would not have appeared if partitions(n) would return a List
> instead of a Stream, but admittedly for big n one might not actually want
> all partitions at once so a lazy aggregate makes sense.
>
> Maybe all this calls for an explicit type FiniteStream (or LazyList) which
> is lazy, but otherwise behaves like List.

I do not think such type make much sense. Either you want to force it
on any pretext or not. If you do not force it, then you gained nothing.
OTOH if you force it then you could use List from the start.

Current design allow you to add trivial functions like

partition_list(n) == complete(partitions(n))

or maybe

partition_list(n) == entries(complete(partitions(n)))

so most user should be able to solve problem to their liking.

--
Waldek Hebisch