Next release

[Waldek Hebisch]

I would like to do new release within a month. Actually,
there is long time without a release so I would like to
do it as fast as possible. But there is a bunch of
issues reported. If possible I would like to get
some new code finished. And there are some structural
changes (in particular releated to OutputForm) that
should go into new release. So, it will probably
take a month.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Ralf Hemmecke]

> But there is a bunch of issues reported.

Obviously almost nobody uses the SF bugtracker.
http://sourceforge.net/p/fricas/bugs/

:-(

I think we should encourage to use it more often.
So, which issues do you mean? Something where I can help?

Ralf

[someone]

Hi,

> I would like to do new release within a month.

That is great news!

> So, it will probably take a month.

Ok, fine.

Are you interested in including the extension and
documentation of matrix slicing done in [1]?
If yes I'd finish the documentation part as
soon as possible.

[1]: https://github.com/raoulb/fricas/tree/twodimarraydoc

[Waldek Hebisch]

someone wrote:
>
> Are you interested in including the extension and
> documentation of matrix slicing done in [1]?
> If yes I'd finish the documentation part as
> soon as possible.
>
> [1]: https://github.com/raoulb/fricas/tree/twodimarraydoc

I am thinking about this. I am a bit worried because
if we would like another form of indexong number of
combinations may get large. So I consider alternative
of sort

LSI => List Segment Integer
IndexType ....
autoCoerce(LSI) -> %
-- maybe just coerce, would need to experiment to find
-- out which one gives better behaviour
autoCoerce(SI) -> %
-- maybe other variants
convert : (%, Integer, Integer) -> LSI
-- extra arguments to represent lower and upper bound
-- for matrix indices, needed if we want complement

and then define


elt : (%, Integer, IndexType)
elt : (%, IndexType, Integer)
elt : (IndexType, Integer)

The idea is that interpreter/compiler magic should convert
what needed to IndexType. I am special casing Integer as
oterwise we would get mismatch with normal array indexing.

It may require tweaking interpreter priorities for choosing
operations.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Waldek Hebisch]

Nr 36 in SF bugtracker got fixed few days ago. It would be
good to do something about nr 37 (ideally we should implement
approximate gcd, minimaly fuzzy version of 'exquo', so that
when gcd decides thet there is common factor 'exquo' should
succeed).

Krystian reported few weird behavoiurs from Spad compiler...

HyperDoc may crash when showing operations from complex
types (more precisely, when type has complex parameters).

Then, there is issue of ')nopile' mode. It is easy to allow
extra semicolons. Current semantics of semicolon is to throw
away computend value, clear but seems useless (except for
signaling error when there are too many semicolons). We
can change this to no-op. Other thing is that semicolons
should be optional after brace. If we allow extra semicolons,
then we may get away with essentially inserting extra
semicolon after closing brace. However still there is
question about:

if b1 then if b2 then {do1}; else {do2}

Namely, with which 'if' has 'else'? Typical sysntax
is that above 'else' goes with second 'if'. But
given that naively inserting semicolons after closing
brace would break

if b1 then if b2 then {do1} else {do2}

where 'else' should go to first 'if'.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Ralf Hemmecke]

I'd like to hear comments from the Aldor experts, that is why I
crosspost to aldor-devel.

https://groups.google.com/d/msg/fricas-devel/OR2CuN7xbhQ/5bDvGkamocQJ

> However still there is
> question about:
>
> if b1 then if b2 then {do1}; else {do2}
>
> Namely, with which 'if' has 'else'? Typical sysntax
> is that above 'else' goes with second 'if'. But
> given that naively inserting semicolons after closing
> brace would break
>
> if b1 then if b2 then {do1} else {do2}
>
> where 'else' should go to first 'if'.

The attached program x.as yields

$aldor -laldor -grun x.as
(0, 0) ..t .;t }.t };t {.t {;t
(0, 1) ..e .;e }.E };E
(1, 0) {.E {;E
(1, 1) {.E {;E

And this is what I would have expected. The "else" always associates to
the most inner "if", if extra braces (as in the last two source lines)
don't change that.

I don't exactly have a pointer into the AUG, but I'd like that the SPAD
rules would be identical to the Aldor rules.

Ralf

[Martin Baker]

On 23/03/14 13:44, Waldek Hebisch wrote:
> I would like to do new release within a month. Actually,
> there is long time without a release so I would like to
> do it as fast as possible. But there is a bunch of
> issues reported. If possible I would like to get
> some new code finished. And there are some structural
> changes (in particular releated to OutputForm) that
> should go into new release. So, it will probably
> take a month.

Waldek,

Please could you add this formatter to the next release:

https://github.com/martinbaker/multivector/blob/master/monospace.spad.pamphlet

The documentation is here:

http://www.euclideanspace.com/maths/standards/program/mycode/output/monospace/

(this contains diagrams that I can't put in the pamphlet)

Its just an optional output formatter with the same structure as the
other formatters. This is designed to output to a monospace device such
as the command line console.

I know you and Ralf are determined to translate i-output.boot to SPAD by
a literal translation directly from the boot code (good luck with that).
This code is completely independent of that, there is no suggestion here
of ever making this the default console output.

The aim of this is to use as the template for more specialist monospace
formatters. It would really help me develop this if you could put it in
the next release.

Also you could look at this as a form of insurance, if you run into
problems doing a literal transpose of i-output.boot, or if the code
produced turns out to be difficult to modify or badly documented, then
you would keep open the option of developing this.

As it stands I think the code here does a reasonable job of displaying
all the various types of expressions in FriCAS, although it is not
perfect. It does not yet handle line wrapping very well, as you pointed
out this is difficult to do, but I'm not convinced that it is as
difficult as you suggest. The program currently generates an array of
Character and then outputs this array to the output device. I could
easily check the width of this array and, if greater than the line
width, choose a place to divide it that does not break apart numbers or
names.

Please could you let me know what you think, if you don't want to
include this in the next release, then I will find something else to
work on.

Martin

[Waldek Hebisch]

Martin Baker wrote:
>
> Please could you add this formatter to the next release:
>
> https://github.com/martinbaker/multivector/blob/master/monospace.spad.pamphlet
>
> The documentation is here:
>
> http://www.euclideanspace.com/maths/standards/program/mycode/output/monospace/
>
> (this contains diagrams that I can't put in the pamphlet)
>
> Its just an optional output formatter with the same structure as the
> other formatters. This is designed to output to a monospace device such
> as the command line console.

Martin, have you tried to run 'src/input/of.input' trough
your formater? When I tried I see a lot of errors.
Sorry for waiting long, but I wanted to have time for
serious tests. Unfortunatly, ATM it is failing the
first test -- 'of.input' contains several constructs,
but only simplest examples of each construct.

> The aim of this is to use as the template for more specialist monospace
> formatters. It would really help me develop this if you could put it in
> the next release.

Well, the tree walkinig part certainly should _not_ be used
as template for other formaters. You use string matching
and ad hoc trick, which lead to errors. Other formaters
(except for Html) were converted to match symbols, that
avoids many problems. Also, decision to put (or not)
parenthesis around a construct should be based on priorities.
The formater responsible for given construct knows
priority of operator (construct) it handles and
is given priority of operator at higher level, so
it has all information needed to make decision about
parenthesis. Trying to do it in other place, like
you do in 'formatBinary' leads to unnecesary complications.
Comparte your formater with TeX of MathML, the logic
to insert parenthesis is much simpler in both of them
(and both use almost the same logic for this).

Concerning 2D formatting it looks that in your formater
simple matrices work OK, but subscripts/superscripts
nested in other constructs cause trouble.

For example:

(exp(x^2/3) + log(1/(x+1)))^10


has problem (this may be because it needs to be split
between lines).


--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Martin Baker]

On 11/04/14 06:06, Waldek Hebisch wrote:
> Martin Baker wrote:
>>
>> Please could you add this formatter to the next release:
>>
>> https://github.com/martinbaker/multivector/blob/master/monospace.spad.pamphlet
>>
>> The documentation is here:
>>
>> http://www.euclideanspace.com/maths/standards/program/mycode/output/monospace/
>>
>> (this contains diagrams that I can't put in the pamphlet)
>>
>> Its just an optional output formatter with the same structure as the
>> other formatters. This is designed to output to a monospace device such
>> as the command line console.
>
> Martin, have you tried to run 'src/input/of.input' trough
> your formater? When I tried I see a lot of errors.
> Sorry for waiting long, but I wanted to have time for
> serious tests. Unfortunatly, ATM it is failing the
> first test -- 'of.input' contains several constructs,
> but only simplest examples of each construct.

I wrote the code in a few days and I did not go on to systematically
test it for all conditions. If I had done so I am reasonably confident
that, by now, it could have been fully working in a reliable way. You
did not seem interested so I did not go on to do so.

I don't know the best thing for me to do? I think that open source
programming should be enjoyable but I don't like working on things that
seem to be competing with what other people are doing and I don't like
reworking low level code when the high level design seems to need
replacing anyway.

I would much rather agree a top level design and that write it properly
once only. But, perhaps you are right, I don't think we will ever agree
on the correct approach to documentation so perhaps we would not agree
on the best design for output formatters either.

>> The aim of this is to use as the template for more specialist monospace
>> formatters. It would really help me develop this if you could put it in
>> the next release.
>
> Well, the tree walkinig part certainly should _not_ be used
> as template for other formaters. You use string matching
> and ad hoc trick, which lead to errors. Other formaters
> (except for Html) were converted to match symbols, that
> avoids many problems. Also, decision to put (or not)
> parenthesis around a construct should be based on priorities.
> The formater responsible for given construct knows
> priority of operator (construct) it handles and
> is given priority of operator at higher level, so
> it has all information needed to make decision about
> parenthesis. Trying to do it in other place, like
> you do in 'formatBinary' leads to unnecesary complications.
> Comparte your formater with TeX of MathML, the logic
> to insert parenthesis is much simpler in both of them
> (and both use almost the same logic for this).

I based this on the html formatter, I deliberately did not change these
because I thought it better if it had the same structure as the other
formatters (I did not know that you had changed all the other formatters
without changing html).

If the tree structure were changed, as I suggested, then the tree
walking would be much more regular and simple. Also, with all this
duplication of code, it is not surprising that the formatters get out of
step. Perhaps, when you add a common category for the formatters, the
common code could be put in that. Although I think it would be better if
some of the processing were done before it even gets to the individual
formatters. For instance parenthesis logic would be done and then just a
binary flag needs to be passed to the formatter to say if parenthesis
needs to be added (this looks like it may be the way SALLI/Aldor is
intended to work).

> Concerning 2D formatting it looks that in your formater
> simple matrices work OK,

I have only put a single horizontal space between cells in a matrix
rather that two used be current formatter. I find a more compact matrix
looks better and, in some cases, it avoids overflow onto two lines.
However these details are all very easy to change when written in high
level code.

> but subscripts/superscripts
> nested in other constructs cause trouble.
>
> For example:
>
> (exp(x^2/3) + log(1/(x+1)))^10
>
>
> has problem (this may be because it needs to be split
> between lines).

This would be very easy to fix. In addition to the rectangle of
characters there is a need to hold a 'base index' then, when
concatenating horizontally, it would make sure that subscripts are
always below the baseline and superscripts are always above the baseline.

Martin

[Waldek Hebisch]

Martin Baker wrote:
>
> On 11/04/14 06:06, Waldek Hebisch wrote:
> >
> > Martin, have you tried to run 'src/input/of.input' trough
> > your formater? When I tried I see a lot of errors.
> > Sorry for waiting long, but I wanted to have time for
> > serious tests. Unfortunatly, ATM it is failing the
> > first test -- 'of.input' contains several constructs,
> > but only simplest examples of each construct.
>
> I wrote the code in a few days and I did not go on to systematically
> test it for all conditions. If I had done so I am reasonably confident
> that, by now, it could have been fully working in a reliable way. You
> did not seem interested so I did not go on to do so.
>
> I don't know the best thing for me to do? I think that open source
> programming should be enjoyable but I don't like working on things that
> seem to be competing with what other people are doing and I don't like
> reworking low level code when the high level design seems to need
> replacing anyway.
>
> I would much rather agree a top level design and that write it properly
> once only. But, perhaps you are right, I don't think we will ever agree
> on the correct approach to documentation so perhaps we would not agree
> on the best design for output formatters either.

Concerning competing: I worked on Fortran output which you did
not want to do. Ralf worked on 'outputTran' which you somewhat
ignored. You wanted change structure of OutputForm. Three
weeks ago substantial change in representation of OutputForm
would break all formatters. Now several formatters access
OutputForm completely via an abstraction layer so in principle
they no longer care how OutputForm is represented. AFAICS
this is complementary to what you wanted to do.

Concerning design: old formater is hard to modify and sometimes
produces ugly output but actual formating errors are rare.
This is critical part of system, so any replacement should
aim at reducing errors. Basic principles (using boxes) are
well-known now. But devil is in details. For example,
formater must have ASCII art for all construct, must calculate
sizes, etc. And there is about 70 constructs to handle.
so there a lot of details which must play well together.
So, reconstructing design of old formater should help
get the details right.

Also, there is question of testing: IME requiering new
formatter to produce exactly the some output as old one
is quite effective at finding problems with new formater
(it may also find problems with old one). Of course,
we may want new formater to produce different output.
But the point is that change should be deliberate.
If new formater from the start produces different output,
than it hard to test: eyeballing thousends lines of output
is not pleasent and it is easy to miss problems. OTOH
once new formater is verfied to produce the same output
as old one (possibly correcting some bugs in old formater)
changes have smaller chance to introduce breakage.

Concerning 'interested': I simply have doubts if your
approach of ignoring details of old formater will
work. Maybe things are realy simple and it will
work. But there is good chance that there are complications
which authors of old formater investigated and can
handle.

Concerning putting formater into new release: I do not
undersand why you insist(ed). I mean, if formater works
fine, then it can go in. But releasing very rough
developement code does not make much sense. You wrote
that this will help you develop it. But you found
good workaround, and anyway rebuilding FriCAS from
sources is not that hard, a lot of folks manage that
and it does not take much time.

>
> >> The aim of this is to use as the template for more specialist monospace
> >> formatters. It would really help me develop this if you could put it in
> >> the next release.
> >
> > Well, the tree walkinig part certainly should _not_ be used
> > as template for other formaters. You use string matching
> > and ad hoc trick, which lead to errors. Other formaters
> > (except for Html) were converted to match symbols, that
> > avoids many problems. Also, decision to put (or not)
> > parenthesis around a construct should be based on priorities.
> > The formater responsible for given construct knows
> > priority of operator (construct) it handles and
> > is given priority of operator at higher level, so
> > it has all information needed to make decision about
> > parenthesis. Trying to do it in other place, like
> > you do in 'formatBinary' leads to unnecesary complications.
> > Comparte your formater with TeX of MathML, the logic
> > to insert parenthesis is much simpler in both of them
> > (and both use almost the same logic for this).
>
> I based this on the html formatter, I deliberately did not change these
> because I thought it better if it had the same structure as the other
> formatters (I did not know that you had changed all the other formatters
> without changing html).

Html formater from the start was different than other -- it was
less regular and more ad hoc. That is why I left it out when
I updated other formaters. I wrote about the change, so you
probably forgot. This time I have updated it to use
'OutputFormTools' but this took significantly more time
than for other formaters (something like 30 minutes per
other formater and abut 3 hours for html). And stil, I did
just minimal change and more should be done.

> If the tree structure were changed, as I suggested, then the tree
> walking would be much more regular and simple. Also, with all this
> duplication of code, it is not surprising that the formatters get out of
> step. Perhaps, when you add a common category for the formatters, the
> common code could be put in that. Although I think it would be better if
> some of the processing were done before it even gets to the individual
> formatters. For instance parenthesis logic would be done and then just a
> binary flag needs to be passed to the formatter to say if parenthesis
> needs to be added (this looks like it may be the way SALLI/Aldor is
> intended to work).

I am not sure different organization would be simpler. There is about
70 construct in OutputForm that need handling. Typically each contructs
needs 1-2 special strings plus some logic. So, each formater has about
100 special strings. Trying to put it into common framework puting
tree walking in category would require each formater to provide
(override) of of order of 100 routines. Possibly more, because
MathML needs special end marker for almost any construct, and
other formaters have slightly differents rules. Common framework
would be forced to call routines from formater at any place
we something special is going on in _any_ of formaters. So,
Formaters based on common framework would have similar size
to current formaters. Actually I looke

Concerning parenthesis, core logic is very simple: just use
'opPrec < prec' test, where 'opPrec' is precision of current
operator and 'prec' is passed from outside. But there are
irregularities (at least fractions need a bit more extra)
and they depend on formater (fraction using '/' has different
property than 2D fraction, and formater may replace one by
another).

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Martin Baker]

On 14/04/14 14:43, Waldek Hebisch wrote:
> Concerning 'interested': I simply have doubts if your
> approach of ignoring details of old formater will
> work. Maybe things are realy simple and it will
> work. But there is good chance that there are complications
> which authors of old formater investigated and can
> handle.

Waldek,

I'm not convinced about this approach, but there is no point in arguing
about it. If I understand correctly you are saying that you and Ralf
have everything in hand so I'm happy to let you get on with it and not
take up any more of your time on this.

Martin

[Waldek Hebisch]

To be clear: I did things which were intedned to help you.
AFAIK Ralf finished 'outputTran' and has no immediate plans
to do more work on 'i-output.boot'. As you requested I planed
to move top level choice of formater (the 'output' function)
to Spad. If you no longer need this, I will probably skip
this past the release. I have no other immediate plans
concerning 'i-output.boot': I consider changes there important,
but they can wait a little.

BTW: It would be nice if you would update HTML formater to
make it more similar to other formaters. You are uniquely
qualified to decide which differences are due to specific
features of HTML and which are accidental.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Martin Baker]

On 15/04/14 01:34, Waldek Hebisch wrote:
> To be clear: I did things which were intedned to help you.
> AFAIK Ralf finished 'outputTran' and has no immediate plans
> to do more work on 'i-output.boot'. As you requested I planed
> to move top level choice of formater (the 'output' function)
> to Spad. If you no longer need this, I will probably skip
> this past the release. I have no other immediate plans
> concerning 'i-output.boot': I consider changes there important,
> but they can wait a little.

Waldek,

My objective is to get to the point where I can do the things on the
projects Sourceforge feature requests, specifically those which require
large changes to type system and interfaces. These are my own personal
interests but I think they would have wider appeal and it would be good
if I could contribute in some way to a resurgence in a popularity of
FriCAS that it deserves.

At the moment the changes these things would require are totally
dependant on you (or possibly Ralf or Bill?). There is no way I could
predict all the consequences of any changes I might make to boot or lisp
code. If you were to stop working on FriCAS, the project would die, and
any work that I did on it would be wasted.

These issues are why I am very keen on your idea to move boot code to
SPAD code since I can more easily contribute to higher level code. I
would want to go much further than just i-output.boot and convert all of
the interpreter as well. Since you and Ralf appeared to me to have
i-output.boot in hand I started to experiment with the main interpreter
loop here:
https://github.com/martinbaker/multivector/blob/master/Interpret.spad

I think, where we differ, is the approach. You want to maximise backward
compatibility and therefore do lots of small incremental steps. I would
like to find an approach that allows me to do what I want a lot faster
even if this means cutting out old stuff. I really like the idea of very
small, lean and high level codebase.

I don't think we are ever going to agree on this and its possible that
what I am trying to do is totally impractical. Therefore it would
probably save a lot of arguing if I just experiment with the interpreter
independently for a while and leave it upto you when you do a new release.

Martin

[Bill Page]

On 15 April 2014 03:54, Martin Baker <ax8...@martinb.com> wrote:
> ...

> I think, where we differ, is the approach. You want to maximise backward
> compatibility and therefore do lots of small incremental steps. I would like
> to find an approach that allows me to do what I want a lot faster even if
> this means cutting out old stuff. I really like the idea of very small, lean
> and high level codebase.
>
> I don't think we are ever going to agree on this and its possible that what
> I am trying to do is totally impractical. Therefore it would probably save a
> lot of arguing if I just experiment with the interpreter independently for a
> while and leave it upto you when you do a new release.
>

Martin,

I doubt that Waldek wants backwards compatibility just for its own
sake. I do not see much arguing going on here, in fact it looks very
cooperative to me. But what you are advocating does seem both
impractical and naive. Computer programming and computer algebra
system development in particular is an "inhuman" activity - something
that most people can do only at the limits of their abilities. if we
want to progress we need to work within these limitations. And another
thing that is limited in open source development is project structure
and co-operation between individuals. Open source development is for
the most part anarchistic, most of the time everybody does their own
thing. Incremental development is one of the most important
strategies for dealing with both these limitations.

On the other hand experimentation is at the heart of open source
development. It is the ability to experiment and the resulting
diversity of approaches that provides something new. So I think it is
great that you are willing to do this and share it with others.

Cheers,
Bill Page.

[Martin Baker]

On 15/04/14 14:56, Bill Page wrote:
> I doubt that Waldek wants backwards compatibility just for its own
> sake.

True but the impression I get with boot code is not just that its got
minimum documentation and minimum typing written in an obscure language
that is hardly documented itself. But, as far as I can make out, the
whole scratchpad project seems to have been used by IBM as a testbed for
research projects which, while being very clever, did not always pay off
in terms of producing maintainable code. Sometimes this sort of code
seems to produce unexpected anomalies and just reproducing that in
another language doesn’t really solve the problem. I get the impression
there are parts (only parts) of the code where it would be better to
rewrite from scratch, it may break some old stuff which relies on the
anomalies but it would be a much firmer foundation for the future.

Also I find SExpressions and their associated pretends quite evil (I
know there are a few cases where we have to use them). One of the
advantages of moving to SPAD is that it is more strongly typed, but
using SExpressions+pretends seems to turn it back into a loosely typed
language again. This may not be a problem for you guys, if you know what
types to expect, but for me it would lead to runtime errors which are
harder to find than compile time errors. I would find it much easier to
go directly to high level code and debug that.

> I do not see much arguing going on here, in fact it looks very
> cooperative to me.

I didn't mean to imply any animosity. I find this sort of discussion
very useful. I just did not want to annoy other people, on the group, if
there is any danger of going over old ground.

> But what you are advocating does seem both
> impractical and naive.

Perhaps but big projects can afford to waste resources, but developers
here seem so thin on the ground that it should be easier to coordinate
better.

Martin

[Ralf Hemmecke]

On 04/15/2014 07:11 PM, Martin Baker wrote:
>> I doubt that Waldek wants backwards compatibility just for its own
>> sake.
>
> True but the impression I get with boot code is not just that its got
> minimum documentation and minimum typing written in an obscure
> language that is hardly documented itself.

Martin, I think, by now you should know that boot code is about to be
removed. The question is: How to do this. There is no good
documentation, so the only option one has to understand what the system
is doing, is to look at the source code. But unfortunately that is BOOT.
:-( Now how can you replace the BOOT code by SPAD code (written from
scratch) if you don't exactly know what specification the SPAD code
should match. If you don't even have a lot of proper test cases, then
it's even harder.

Suppose, we simply remove i-output.boot and take something (in SPAD)
that you give us. I'd be much in favour of that. You need not copy
i-output.boot, but there is one condition for you: after removing that
i-output.boot, FriCAS should still work at least as good as before.

Can you give an estimate *when* you will have such a SPAD file? Note
that the goal is to remove i-output.boot, not just adding a new SPAD file.

> I get the impression there are parts (only parts) of the code where
> it would be better to rewrite from scratch, it may break some old
> stuff which relies on the anomalies but it would be a much firmer
> foundation for the future.

Nobody stops you from writing such code. No, honestly, there is nothing
wrong with your approach. Do it. Show us the code and explain which part
of FriCAS will be better and which boot files can be removed by your
code. (Your patch can actually already contain the removal of that boot
code.) Add sufficient test cases so that it would be easy for a reviewer
of your code that (1) it does what it claims to do and (2) shows that
the old system is only changed at deliberate places.

> Also I find SExpressions and their associated pretends quite evil (I
> know there are a few cases where we have to use them).

There are always parts of the system that is relatively low-level.
SExpression is such a domain. I would actually count it as not too far
away from Lisp. So if you don't have to, then don't use SExpression. Go
with Tree(S) instead.

> One of the advantages of moving to SPAD is that it is more strongly
> typed, but using SExpressions+pretends seems to turn it back into a
> loosely typed language again. This may not be a problem for you guys,
> if you know what types to expect, but for me it would lead to runtime
> errors which are harder to find than compile time errors. I would
> find it much easier to go directly to high level code and debug
> that.

Same for me. I neither like "pretend" nor SExpression nor InputForm and
all that low-level stuff. But we don't yet have something cleaner.

Why don't you just write your own abstraction layer on top of a few
basic domains and then never ever again use "pretend" and low-level
domains, but rather use your abstraction layer?

That would give you nice types and wouldn't stop you from implementing
stuff that you like to implement.

I'd really be happy if we could replace our graphics engine with
something better. I have the impression that you have the potential to
write a new interactive system. It wouldn't matter if we have two
systems. Your code does not even have to live in the SVN archive. I'm
currently working on the book and all of that is just local on my laptop
and partly on my github account. When Waldek commits new code, I often
rebase my branch on top of the new trunk. That's usually not a problem
and while compiling I immediately see whether Waldek's code breaks
something in my branch.

We are basically all working on orthogonal stuff. So what really stops
you from doing what you want? Just because Waldek seemingly doesn't want
big rewrites, there is no point in just doing what you want. If after
some time (maybe several months) you show that your code is superior or
your code simply helps *you* to do your work or to just be happy about
it, then go on.

> Perhaps but big projects can afford to waste resources, but
> developers here seem so thin on the ground that it should be easier
> to coordinate better.

The code is there, start working on what your really like to do. You can
even fork the code (actually with git, you (and me) have already done
so) and be the master of your own fork. You can always include the code
from FriCAS into your fork. Continue and don't wait for someone to tell
you what you should do or whether your code will ever be included into
FriCAS. That doesn't matter. If the code is good for your, that should
suffice to write it.

Ralf

[Waldek Hebisch]

Martin Baker wrote:
>
> Also I find SExpressions and their associated pretends quite evil (I
> know there are a few cases where we have to use them). One of the
> advantages of moving to SPAD is that it is more strongly typed, but
> using SExpressions+pretends seems to turn it back into a loosely typed
> language again. This may not be a problem for you guys, if you know what
> types to expect, but for me it would lead to runtime errors which are
> harder to find than compile time errors. I would find it much easier to
> go directly to high level code and debug that.

Just to make clear what the direction is: formaters in FriCAS 1.2.2
(Formula, TeX, Texmacs, MathML and HTML) contain 37 pretends and
54 '$Lisp' calls. Trunk today has 13 pretends in formaters + OFTOOL
and 31 '$Lisp' calls. AFAICS all pretend and '$Lisp' calls related
to OutputForm are in OFTOOL. Most of remaining '$Lisp' calls are
debugging printouts. So, the result is far from ideal, but IMHO
we got significant improvement with relatively little effort.

Also, I when SExpression is openly used, then it should be enough
to use exported interface of SExpression, without any pretends.
Pretends appear when we want to have stronger typing having
SExpression as representation.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Bill Page]

On 15 April 2014 13:11, Martin Baker <ax8...@martinb.com> wrote:
> On 15/04/14 14:56, Bill Page wrote:
>>
>> I doubt that Waldek wants backwards compatibility just for its own
>> sake.
>
>
> True but the impression I get with boot code is not just that its got
> minimum documentation and minimum typing written in an obscure
> language that is hardly documented itself.

There is nothing particularly difficult or obscure about BOOT. It
started out as a preprocessor for Lisp of the kind that was originally
envisaged by the developers of Lisp but never fully implemented.
Instead Lisp programmers adopted s-expressions for both data and
fuctions.

http://en.wikipedia.org/wiki/M-expression

The original developers of Axiom wanted something higher level but
still close to Lisp as a stepping stone to a higher level language
(SPAD). Inventing such domain-specific languages is actually one of
the primary development paradigms for large systems based on Lisp.
Since then several other variants of m-expression syntax for Lisp have
appeared.

Axiom ended up with a mixture of Lisp, BOOT, and SPAD but the use of
BOOT was not universally accepted even by the Axiom developers and
this was still an important issue for Tim Daly at the start of the
Axiom open source project. One of the priorities of that project was
to drop BOOT and adopt the generated Lisp code as the primary source
code as the uniform "internal" language of the Axiom project. (I.e.
going in the opposite direction to what you propose.)

Within the original Axiom project once SPAD was (more or less)
stablized there was a desire to incrementally upgrade BOOT code to
SPAD where that was easily possible. At about this same time a
decision was taken to develop Aldor, a replacement for SPAD that in
principle would make it easier to replace all of Lisp and BOOT with
higher level code. Unfortunately before this goal could be achieved
the developers at IBM where forced to make their research project into
a commercial venture which failed not long after. (To make a long
story short.)

> But, as far as I can make out, the whole
> scratchpad project seems to have been used by IBM as a testbed for research
> projects which, while being very clever, did not always pay off in terms of
> producing maintainable code. Sometimes this sort of code seems to produce
> unexpected anomalies and just reproducing that in another language doesn't
> really solve the problem. I get the impression there are parts (only parts)
> of the code where it would be better to rewrite from scratch, it may break
> some old stuff which relies on the anomalies but it would be a much firmer
> foundation for the future.
>

You might not be too surprised to learn that this is also a good
summary of Reduce, Maxima, Maple, Mathematica and several other
popular computer algebra systems.

> Also I find SExpressions and their associated pretends quite evil (I know
> there are a few cases where we have to use them). One of the advantages
> of moving to SPAD is that it is more strongly typed, but using
> SExpressions+pretends seems to turn it back into a loosely typed
> language again.

I think that calling SExpressions "loosely typed" is a misnomer. In
fact SExpressions as a domain in Axiom/FriCAS are as strongly typed as
any other object, it is just that they happen to correspond to the
underlying data structure used by Lisp. It is pretend itself that is
the (necessary) evil.

> This may not be a problem for you guys, if you know what types to
> expect, but for me it would lead to runtime errors which are harder to find
> than compile time errors. I would find it much easier to go directly to high
> level code and debug that.
>

I think everyone here agrees with you.

>
>> I do not see much arguing going on here, in fact it looks very
>> cooperative to me.
>
>
> I didn't mean to imply any animosity. I find this sort of discussion very
> useful. I just did not want to annoy other people, on the group, if there is
> any danger of going over old ground.
>

Yes it is old ground indeed but still worth going over. :)

>
>> But what you are advocating does seem both
>> impractical and naive.
>
>
> Perhaps but big projects can afford to waste resources, but developers here
> seem so thin on the ground that it should be easier to coordinate better.
>

Why do you think it should be easier to coordinate better?

[Martin Baker]

On 15/04/14 22:44, Bill Page wrote:
> Yes it is old ground indeed but still worth going over. :)

I agree, I find it interesting to hear about the history of the project
and I think it does help to know why things were done in a certain way.

Some introductions to the program mention the IBM origins but they
sometimes seem to have an 'agenda'. I find it interesting to hear both
sides of the controversies in an unbiased way as you have written here.

Martin

[Ralf Hemmecke]

> Some introductions to the program mention the IBM origins but they
> sometimes seem to have an 'agenda'. I find it interesting to hear both
> sides of the controversies in an unbiased way as you have written here.

Hi Bill,

can you put your knowledge into the axiom-wiki so that it is easy to
refer to it?

Thanks
Ralf

[someone]

Hi,

> I am thinking about this. I am a bit worried because
> if we would like another form of indexong number of
> combinations may get large.

That is of course true.

> So I consider alternative
> of sort
>
> LSI => List Segment Integer
> IndexType ....
> autoCoerce(LSI) -> %
> -- maybe just coerce, would need to experiment to find
> -- out which one gives better behaviour
> autoCoerce(SI) -> %
> -- maybe other variants
> convert : (%, Integer, Integer) -> LSI
> -- extra arguments to represent lower and upper bound
> -- for matrix indices, needed if we want complement

> and then define
>
>
> elt : (%, Integer, IndexType)
> elt : (%, IndexType, Integer)
> elt : (IndexType, Integer)

To some larger part I like this approach.
We could port that to other indexable structures
not just matrices. (At least vectors should work too.)

Can we maybe even get rid of Integer?
Integers should then be coerced to IndexType things.

> The idea is that interpreter/compiler magic should convert
> what needed to IndexType. I am special casing Integer as
> oterwise we would get mismatch with normal array indexing.

Hmm.
Why would it not work if we index Objects *only* by IndexType
elements and Integers are (can be coerced to) that Domain?

> It may require tweaking interpreter priorities for choosing
> operations.

Ok.


To make the other point, the code I was talking about is complete
and works by now. This unified approach seems to be superior and
better from the software design view, but does not exist yet.

Therefore I'd still vote for inclusion and improve later if we like.

[Waldek Hebisch]

For setters this may work. But indexing via IndexType returns
matrix, while when both indices are integers we should
get the element. Overloading on return type may sometimes
help, but I am affraid that having coercion from integers
to IndexType would lead to nasty surprizes.

> To make the other point, the code I was talking about is complete
> and works by now. This unified approach seems to be superior and
> better from the software design view, but does not exist yet.
>
> Therefore I'd still vote for inclusion and improve later if we like.

I am now testing your code. I had to update it for 's/setelt/setelt!/'
change. Assuming it works OK I am going to include it in the
release.
--
Waldek Hebisch
heb...@math.uni.wroc.pl

[someone]

Hi,

> Why would it not work if we index Objects *only* by IndexType
> > elements and Integers are (can be coerced to) that Domain?
>
> For setters this may work. But indexing via IndexType returns
> matrix, while when both indices are integers we should
> get the element.

Ok. This way on can have both, single entries and 1x1 Matrices
depending on context and use case.

> Overloading on return type may sometimes
> help, but I am affraid that having coercion from integers
> to IndexType would lead to nasty surprizes.

Probably right and not worth the troubles.

> > To make the other point, the code I was talking about is complete
> > and works by now. This unified approach seems to be superior and
> > better from the software design view, but does not exist yet.
> >
> > Therefore I'd still vote for inclusion and improve later if we like.
>
> I am now testing your code.

Thanks. Let me know about any issues you find.

> I had to update it for 's/setelt/setelt!/' change.

Ok.

> Assuming it works OK I am going to include it in the release.

Perfect. I'll try to finish the documentation tonight.

[someone]

Hi,

> Assuming it works OK I am going to
> include it in the release.

I updated the code to work on top of current Fricas.

https://github.com/raoulb/fricas/commits/twodimarraydoc


There is some more documentation now, I hope this
is sufficient to explain the new features. I split
it into two commits in case you decide not to include
the slicing functions:

https://github.com/raoulb/fricas/commit/9c8748c623a46543e847ee3320526717f42aafc3
https://github.com/raoulb/fricas/commit/56985e962e2e9d8f2183f5815cde3032bb5fc4e2

[Waldek Hebisch]

someone wrote:
>
> > Assuming it works OK I am going to
> > include it in the release.
>
> I updated the code to work on top of current Fricas.
>
> https://github.com/raoulb/fricas/commits/twodimarraydoc
>

I have some doubts about handling empty segments. First,
'subMatrix(m. lr, ur, lc, uc)' performs the same operations
as 'm(lr..ur, lc..uc)' so it would be good if they acceted
the same set of inputs. But after your patch they do not agree.
Second, FriCAS has tradition of raising errors on suspicious
input. In such spirit 'subMatrix' allows empty segments
because it is necessary to generate empty slice, but
only when lower bound is 1 + upper bound, which seem to
cover main uses but still can catch a lot of errors.
I admit that lower bound <= 1 + upper bound is somewhat
arbitrary, but it is not clear for me if arbitary empty
segments are really useful.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Ralf Hemmecke]

On 04/28/2014 09:57 PM, Waldek Hebisch wrote:
> someone wrote:
>>
>>> Assuming it works OK I am going to
>>> include it in the release.
>>
>> I updated the code to work on top of current Fricas.
>>
>> https://github.com/raoulb/fricas/commits/twodimarraydoc
>>
>
> I have some doubts about handling empty segments.

Why?

We have the empty matrix already.

(3) -> m := matrix([]$List(List(Integer)))

(3)
Type:
Matrix(Integer)
(4) -> m+m

(4)
Type:
Matrix(Integer)
(5) -> n := matrix([[],[],[]]$List(List(Integer)))

++
||
(5) ||
||
++
Type:
Matrix(Integer)

Not that I think that is useful, but there is no clash to the current code.

Ralf

[Waldek Hebisch]

Ralf Hemmecke wrote:
>
> On 04/28/2014 09:57 PM, Waldek Hebisch wrote:
> > someone wrote:
> >>
> >>> Assuming it works OK I am going to
> >>> include it in the release.
> >>
> >> I updated the code to work on top of current Fricas.
> >>
> >> https://github.com/raoulb/fricas/commits/twodimarraydoc
> >>
> >
> > I have some doubts about handling empty segments.
>
> Why?

With the patch:

(1) -> m := matrix([[1, 2, 3], [4, 5, 6]])

+1 2 3+
(1) | |
+4 5 6+
Type: Matrix(Integer)
(2) -> subMatrix(m, 2, 1, 1, 2)

(2)
Type: Matrix(Integer)
(3) -> subMatrix(m, 3, 1, 1, 2)

>> Error detected within library code:
subMatrix: bad row indices

(3) -> m(2..1, 1..2)

(3)
Type: Matrix(Integer)
(4) -> m(3..1, 1..2)

(4)
Type: Matrix(Integer)

In current trunk line 4 signals error. For consistency it
would be good to get the same result from (2) and (4).

BTW: Empty matrices are useful to avoid special cases. For
example, from computation you may get empty system of equations.
It is convenient to blindly pass it to 'solve' and get right
answer. Of course, 'solve' need to special case empty system.
But we avoid special cases in callers.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[someone]

> With the patch:
>
> (1) -> m := matrix([[1, 2, 3], [4, 5, 6]])
>
> +1 2 3+
> (1) | |
> +4 5 6+
> Type:
> Matrix(Integer) (2) -> subMatrix(m, 2, 1, 1, 2)
>
> (2)
> Type:
> Matrix(Integer) (3) -> subMatrix(m, 3, 1, 1, 2)
>
> >> Error detected within library code:
> subMatrix: bad row indices
>
> (3) -> m(2..1, 1..2)
>
> (3)
> Type:
> Matrix(Integer) (4) -> m(3..1, 1..2)
>
> (4)
> Type:
> Matrix(Integer)
>
> In current trunk line 4 signals error. For consistency it
> would be good to get the same result from (2) and (4).

If you like, we can of course change that.

But we could also turn around the view point: if 2..1 does
not signal an error, why should 3..1 be inconsistent with
the "rule"? Depends on what we take as the rule.

Signaling errors is in general a good idea, hence this might
give a hint on which rule to take. OTOH we need to make sure
that no code ever will use the "feature" of 2..1 and run
into trouble while blindly assuming it works the same for 3..1.
This might give a hint on why to handle empty slices like I did.

For consistency (which I agree is also important) one could potentially
change the behavior of subMatrix, given this causes not problems on
other spots.

I'm not really sure which point here is optimal. Maybe one should
even look at how matlab/octave and python/numpy handle such
slices? This could give consistency across systems and potentially
match user expectations best?

> BTW: Empty matrices are useful to avoid special cases.

This is very true once you look at "numerical" algorithms
working with matrices and slices.

[Waldek Hebisch]

someone wrote:
>
> > With the patch:
> >
> > (1) -> m := matrix([[1, 2, 3], [4, 5, 6]])
> >
> > +1 2 3+
> > (1) | |
> > +4 5 6+
> > Type:
> > Matrix(Integer) (2) -> subMatrix(m, 2, 1, 1, 2)
> >
> > (2)
> > Type:
> > Matrix(Integer) (3) -> subMatrix(m, 3, 1, 1, 2)
> >
> > >> Error detected within library code:
> > subMatrix: bad row indices
> >
> > (3) -> m(2..1, 1..2)
> >
> > (3)
> > Type:
> > Matrix(Integer) (4) -> m(3..1, 1..2)
> >
> > (4)
> > Type:
> > Matrix(Integer)
> >
> > In current trunk line 4 signals error. For consistency it
> > would be good to get the same result from (2) and (4).
>
> If you like, we can of course change that.

I think that for now we should singnal error in upper bound
is smaller than lower + 1.


--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Ralf Hemmecke]

On 04/28/2014 11:16 PM, Waldek Hebisch wrote:
> With the patch:
>
> (1) -> m := matrix([[1, 2, 3], [4, 5, 6]])
>
> +1 2 3+
> (1) | |
> +4 5 6+
> Type: Matrix(Integer)
> (2) -> subMatrix(m, 2, 1, 1, 2)
>
> (2)
> Type: Matrix(Integer)
> (3) -> subMatrix(m, 3, 1, 1, 2)
>
> >> Error detected within library code:
> subMatrix: bad row indices
>
> (3) -> m(2..1, 1..2)
>
> (3)
> Type: Matrix(Integer)
> (4) -> m(3..1, 1..2)
>
> (4)
> Type: Matrix(Integer)
>
> In current trunk line 4 signals error. For consistency it
> would be good to get the same result from (2) and (4).

Well, it actually depends, whether (2) and (4) *are* different.
Do we see a Segment "a..b by c" just

(A) as the triple (a,b,c) or
(B) as the set of values {a, a+c, a+2c, ... a+nc} with n such that
a+nc<=b?

The interface of Segment suggests the first, because from (B) it would
be generally impossible to extract high(s) from a segment s. So in this
sense, Waldek is right to treat (2) and (4) in the same way.

Personally, it would also be more intuitive to me that m(3..1,1..2)
behaves just the same as m(2..1,1..2), because I would naturally look at
it as if it were written m(expand(3..1), expand(1..2)).

So technically, Waldek is right, but then we should also think about

(6) -> sum(1/n, n=1..-1)

(6) 0

Well, maybe that example is a little different, but still, it doesn't
evaluate 1/n if the interval is empty. Perhaps, I should have used this.

(9) -> li := [7]

(9) [7]
Type:
List(PositiveInteger)
(10) -> li.1

(10) 7
Type:
PositiveInteger
(11) -> sum(li.n, n=1..-1)
There are 3 exposed and 1 unexposed library operations named li
having 1 argument(s) but none was determined to be applicable.
Use HyperDoc Browse, or issue
)display op li
to learn more about the available operations. Perhaps
package-calling the operation or using coercions on the arguments
will allow you to apply the operation.

Cannot find a definition or applicable library operation named li
with argument type(s)
Variable(n)

Perhaps you should use "@" to indicate the required return type,
or "$" to specify which version of the function you need.


The last sum also gives this error if the upper bound is 1. That looks
like a bug.

Ralf

[Waldek Hebisch]

I noticed another little quirk. In ug08.htex we have:

> By using the functions \spadfun{rowSlice} and \spadfun{colSlice} it is
> possible to obtain for a given matrix two special slicing objects that
> when used will select all elements along a row or column respectively.

which suggests that using rowSlice will give rows. But in fact
it gives columns -- rowSlice varies row indices which slects a
column. Maybe w should switch the names? Or make clear what
they are doing.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[someone]

Hi,

> > By using the functions \spadfun{rowSlice} and \spadfun{colSlice} it
> > is possible to obtain for a given matrix two special slicing
> > objects that when used will select all elements along a row or
> > column respectively.
>
> which suggests that using rowSlice will give rows. But in fact
> it gives columns -- rowSlice varies row indices which slects a
> column. Maybe w should switch the names? Or make clear what
> they are doing.

This is correct, the names are maybe bit misleading. "rowSlice"
will give a "row slicer", an object one should put into
the row slot of the element access pair.

I think the current text you put into hyperdoc is ok.

[someone]

Hi,

> > If you like, we can of course change that.
>
> I think that for now we should singnal error in upper bound
> is smaller than lower + 1.

Ok.

Thanks for including my code into Fricas.

[someone]

Hi,

> The interface of Segment suggests the first, because from (B) it would
> be generally impossible to extract high(s) from a segment s. So in
> this sense, Waldek is right to treat (2) and (4) in the same way.
>
> Personally, it would also be more intuitive to me that m(3..1,1..2)
> behaves just the same as m(2..1,1..2), because I would naturally look
> at it as if it were written m(expand(3..1), expand(1..2)).

Isn't that one of the two options I spoke about?