list(3.. by 2)

[Ralf Hemmecke]

I guess
1) the "by 2" should not be ignored.
2) "2.. by 2" should be counted as a correct expression for a
UniversalSegment.

```
(12) -> l := [a,b,c,d,e,f,g,h,i,j]

(12) [a, b, c, d, e, f, g, h, i, j]
Type:
List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
(15) -> l(2..7)

(15) [b, c, d, e, f, g]
Type:
List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
(16) -> l(2..7 by 2)

(16) [b, c, d, e, f, g]
Type:
List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
(17) -> l(2.. by 2)
There are no library operations named BY having 1 argument(s) though
there are 1 exposed operation(s) and 0 unexposed operation(s)
having a different number of arguments. Use HyperDoc Browse, or
issue
)what op BY
to learn what operations contain " BY " in their names, or issue
)display op BY
to learn more about the available operations.

Cannot find a definition or applicable library operation named BY
with argument type(s)
PositiveInteger

Perhaps you should use "@" to indicate the required return type,
or "$" to specify which version of the function you need.

(17) -> l((2..) by 2)

(17) [b, c, d, e, f, g, h, i, j]
Type:
List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
```

Clearly, the problem lies here
https://github.com/fricas/fricas/blob/master/src/algebra/aggcat.spad#L1658
```
elt(x : %, i : UniversalSegment(Integer)) ==
l := low(i) - minIndex x
l < 0 => error "index out of range"
not hasHi i => copy(rest(x, l::NonNegativeInteger))
(h := high(i) - minIndex x) < l => empty()
first(rest(x, l::NonNegativeInteger), (h - l +
1)::NonNegativeInteger)
```
However, it is not so clear how the stepsize can be taken into account,
given that a generic implementation should work for finite (List) and
infinite (Stream) structures.

The same applies for the delete function.

Should we rather move that function into the respective domains and
implement it properly there? Or explicitly forbid (in the ++ docstrng
the "by n" in the UniversalSegment?

Ralf

[Waldek Hebisch]

On Tue, May 24, 2022 at 12:26:03PM +0200, Ralf Hemmecke wrote:
> I guess
> 1) the "by 2" should not be ignored.
> 2) "2.. by 2" should be counted as a correct expression for a
> UniversalSegment.
>
> ```
> (12) -> l := [a,b,c,d,e,f,g,h,i,j]
>
> (12) [a, b, c, d, e, f, g, h, i, j]
> Type:
> List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
> (15) -> l(2..7)
>
> (15) [b, c, d, e, f, g]
> Type:
> List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
> (16) -> l(2..7 by 2)
>
> (16) [b, c, d, e, f, g]
> Type:
> List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
> (17) -> l(2.. by 2)
> There are no library operations named BY having 1 argument(s) though
> there are 1 exposed operation(s) and 0 unexposed operation(s)
> having a different number of arguments.

This is parsing issue.

> (17) -> l((2..) by 2)
>
> (17) [b, c, d, e, f, g, h, i, j]
> Type:
> List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
> ```
>
> Clearly, the problem lies here
> https://github.com/fricas/fricas/blob/master/src/algebra/aggcat.spad#L1658
> ```
> elt(x : %, i : UniversalSegment(Integer)) ==
> l := low(i) - minIndex x
> l < 0 => error "index out of range"
> not hasHi i => copy(rest(x, l::NonNegativeInteger))
> (h := high(i) - minIndex x) < l => empty()
> first(rest(x, l::NonNegativeInteger), (h - l +
> 1)::NonNegativeInteger)
> ```
> However, it is not so clear how the stepsize can be taken into account,
> given that a generic implementation should work for finite (List) and
> infinite (Stream) structures.
>
> The same applies for the delete function.
>
> Should we rather move that function into the respective domains and
> implement it properly there? Or explicitly forbid (in the ++ docstrng the
> "by n" in the UniversalSegment?

Well there are two possibilities:

1) In generic implementation signal error when 'by' is present
2) add apropriate primitive operation(s) to category and implement
them in specific domains domains

Signaling error has disadvantage thaty putting restrictions in
documentation adds bulk and distract users ("do this restriction
affects my use case?"). OTOH if users hit undocumented restriction,
they will be dissatisfied.

I am not sure how important is 'elt' with 'by'. Admitedly, I make
little use of 'elt' with segments, but I consider it an important
feature. If 'elt' with 'by' is important enough to explicitely
spell out restricition in documentation, then I think it is important
enough to implement properly (without restriction).

--
Waldek Hebisch

[Ralf Hemmecke]

>> List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j])) (17) -> l(2.. by
>> 2) There are no library operations named BY having 1 argument(s)
>> though there are 1 exposed operation(s) and 0 unexposed
>> operation(s) having a different number of arguments.
>
> This is parsing issue.

Is it possible that you can fix this?

> Well there are two possibilities:
>
> 1) In generic implementation signal error when 'by' is present 2) add
> apropriate primitive operation(s) to category and implement them in
> specific domains domains

First investigation says that we could have a proper implementation in
StreamAggregate, if the respective domains implement something like
multisect.

For Stream itself we could just move it from
StreamTaylorSeriesOperations.

https://github.com/fricas/fricas/blob/master/src/algebra/sttaylor.spad#L445

In fact, looking at the functions in StreamTaylorSeriesOperations, I
wonder why not many of them could live directly in Stream. The arguments
are mostly bound to "Stream" anyway. Maybe open for discussion.

Implementation of multisect for List, is probably not that hard, except
that one must take care of cyclic lists.

It seems that only Stream and List are the most important instances for
this elt function, although there are more like AssociationList.

So what do you think about implementation of multisect in List and Stream?

> Signaling error has disadvantage thaty putting restrictions in
> documentation adds bulk and distract users ("do this restriction
> affects my use case?"). OTOH if users hit undocumented restriction,
> they will be dissatisfied.

> I am not sure how important is 'elt' with 'by'.

Oh, probably not that important since nobody has asked for it for so
many years, but I came across the Mathematica "span" syntax "a;;b;;s".
https://reference.wolfram.com/language/ref/Span.html
And there is also list slicing in Python ... "[start : stop : steps]".
https://www.geeksforgeeks.org/python-list-comprehension-and-slicing/
I thought we can just make things work properly in FriCAS.

> Admitedly, I make little use of 'elt' with segments, but I consider
> it an important feature. If 'elt' with 'by' is important enough to
> explicitely spell out restricition in documentation, then I think it
> is important enough to implement properly (without restriction).

I'd prefer for proper implementation.

We also seem to have a bug (although I question that al(1..1) would be
anything useful.

(22) -> AL ==> AssociationList(String, List String)

(23) -> al := empty()$AL

(23) table()
Type: AssociationList(String,List(String))
(24) -> al."fire" := ["red", "orange", "yellow"]

(24) ["red", "orange", "yellow"]
Type: List(String)
(25) -> al."water" := ["blue"]
(25) ["blue"]
Type: List(String)
(32) -> al(1..1) -- doesn't stop
C-c C-c
>> System error:
Interactive interrupt at #x534C5682.

Ralf

[Waldek Hebisch]

On Sat, May 28, 2022 at 02:55:50PM +0200, Ralf Hemmecke wrote:
>
> We also seem to have a bug (although I question that al(1..1) would be
> anything useful.
>
> (22) -> AL ==> AssociationList(String, List String)
>
> (23) -> al := empty()$AL
>
> (23) table()
> Type: AssociationList(String,List(String))
> (24) -> al."fire" := ["red", "orange", "yellow"]
>
> (24) ["red", "orange", "yellow"]
> Type: List(String)
> (25) -> al."water" := ["blue"]
> (25) ["blue"]
> Type: List(String)
> (32) -> al(1..1) -- doesn't stop
> C-c C-c
> >> System error:
> Interactive interrupt at #x534C5682.

The problem here is really with 'first'. AssociationList uses
implementation of 'first' from LinearAggregate:

first(x, n) == x(minIndex(x)+(0..(n-1)))

that is 'first(x, 2)' calls 'elt(x, 1..2)'. But 'elt'
is implemented in StreamAggregate in terms of 'first',
so that 'elt(x, 1..2)' calls 'first(x, 2)'. So we get
infinite recursion. Due to tail call optimization
this works as infinite loop.

Note that 'first(al, 2)' or 'elt(al, 2)' is sensible.

StreamAggregate has two childrens: ListAggregate and
LazyStreamAggregate. ATM all domains which have
StreamAggregate are also of children categories.
Also there are only two domains having LazyStreamAggregate:
Stream and Sequence. Sequence inherits implementation from
stream. So, it makes sense to remove implementation of 'elt'
from 'StreamAggregate' and add implementations to
ListAggregate and Stream. In ListAggregate we can use list
operations to implement it. In Stream we can give proper
lazy implementation (we need 'delay' for this so we can not
do this in LazyStreamAggregate).

--
Waldek Hebisch

[Ralf Hemmecke]

On 28.05.22 17:44, Waldek Hebisch wrote:
> On Sat, May 28, 2022 at 02:55:50PM +0200, Ralf Hemmecke wrote:
>>
>> We also seem to have a bug (although I question that al(1..1)
>> would be anything useful.
>>
>> (22) -> AL ==> AssociationList(String, List String)
>>
>> (23) -> al := empty()$AL
>>
>> (23) table() Type: AssociationList(String,List(String)) (24) ->
>> al."fire" := ["red", "orange", "yellow"]
>>
>> (24) ["red", "orange", "yellow"] Type: List(String) (25) ->
>> al."water" := ["blue"] (25) ["blue"] Type: List(String) (32) ->
>> al(1..1) -- doesn't stop C-c C-c
>>>> System error:
>> Interactive interrupt at #x534C5682.
>
> The problem here is really with 'first'. AssociationList uses
> implementation of 'first' from LinearAggregate:
>
> first(x, n) == x(minIndex(x)+(0..(n-1)))
>
> that is 'first(x, 2)' calls 'elt(x, 1..2)'. But 'elt' is
> implemented in StreamAggregate in terms of 'first', so that 'elt(x,
> 1..2)' calls 'first(x, 2)'. So we get infinite recursion. Due to
> tail call optimization this works as infinite loop.
>
> Note that 'first(al, 2)' or 'elt(al, 2)' is sensible.

Well, I do not think so. What would be the specification? Get the second
element of the internal representation of al? Who can predict whether
this will be ["fire",...] or ["water",...]. It becomes only sensible
IMHO if I do

KV==>Record(key: String, entry: List String)
(entries(al)@List(KV)).2,

but then we are talking about List anyway.

The other thing is that I do something like

[foo(al.n) for n in 1..#al]

which should/could be done via

[foo x for x in entries(al)@List(KV)

Well, existence of that elt does not hurt, but I would somehow question
"sensible" unless someone gives me a good use case that I obviously now
overlook.

> So, it makes sense to remove implementation of 'elt' from
> 'StreamAggregate' and add implementations to ListAggregate and

> Stream In ListAggregate we can use list operations to implement it.

OK, that's what I wanted to hear.

> In Stream we can give proper lazy implementation (we need 'delay'
> for this so we can not do this in LazyStreamAggregate).

Yes, that was exactly the problem that I faced when I tried to implement
it generically.

I try to prepare a pull-request.

Ralf

[Waldek Hebisch]

Pairs with non-present key are added at the beginning. And important
use case is when pairs are prepended. After prepending k pairs
'rest(x, k)' gives you back original association list. But
one may need to do some processing (say cleanup) for dropped
elements. 'first(x, k)' is convenient way to get those elements.
I tend to write such things using 'first(x)', 'rest(x)' and
iteration but other folks may prefer 'first(x, k)'.

--
Waldek Hebisch

[Waldek Hebisch]

On Tue, May 24, 2022 at 12:26:03PM +0200, Ralf Hemmecke wrote:

> I guess
> 1) the "by 2" should not be ignored.
> 2) "2.. by 2" should be counted as a correct expression for a
> UniversalSegment.

<snip>

> List(OrderedVariableList([a,b,c,d,e,f,g,h,i,j]))
> (17) -> l(2.. by 2)
> There are no library operations named BY having 1 argument(s) though
> there are 1 exposed operation(s) and 0 unexposed operation(s)
> having a different number of arguments. Use HyperDoc Browse, or
> issue
> )what op BY
> to learn what operations contain " BY " in their names, or issue
> )display op BY
> to learn more about the available operations.
>
> Cannot find a definition or applicable library operation named BY
> with argument type(s)
> PositiveInteger
>
> Perhaps you should use "@" to indicate the required return type,
> or "$" to specify which version of the function you need.

As I wrote, this was parsing problem. It is fixed now.

However, looking at 'cparse.boot' I think that is fundamentally
wrong. Or, to say it differently it works on different principle
than old parser which is some (somewhat uncommon) cases leads
to different (IMO wrong) parse. Namely, old parser is based on
opertor priorities. This agrees with common way to parse
expressions. 'cparse.boot' is organized differently, using
hierarhy of functions to encode priorites. However, with
many priority levels and complex priorites (left and right
times Led and Nud) one would get very comples hierarhy.
'cparse.boot' plays clever tricks to essentially compress
hierarhy to much simpler thing, but unfortunatly, this does
not agree with priorities.

I guess we should give higher priority to replacing 'cparse.boot'
by old parser...

--
Waldek Hebisch