New release

[Waldek Hebisch]

I would like to do new release in June. I would like to include
Qian work on HyperDoc and the FriCAS book, that is main thing
that needs to be resolved before release.

--
Waldek Hebisch

[Ralf Hemmecke]

On 6/10/24 20:34, Waldek Hebisch wrote:
> I would like to do new release in June. I would like to include
> Qian work on HyperDoc and the FriCAS book, that is main thing
> that needs to be resolved before release.

I am looking at Qian's stuff.

Unfortunately, the new release will break the FriCAS-Aldor interface if
nothing is done about it. I'm not really happy about that.

Ralf

[Waldek Hebisch]

For release we can revert the change to algebra, ATM nothing depends
on this change. But this is important change, and allows better
structuring of integration code. So either we allow Aldor to block
developement of algebra (and I do not want to do this) or somebody
(and I am affraid only Peter can do this) changes Aldor to remove
current limitation or interface will be broken in the future.

--
Waldek Hebisch

[Qian Yun]

I'd like you to reconsider updating config/* files.

We updated them to 2014 version in 2017.
I guess you didn't like "-unknown-" in the target triplet,
so you did manual modifications to remove it.
But with newer config/* files, this is no longer a problem,
now the system will be identified as "-pc-".

- Qian

[Qian Yun]

I can commit the patch in "[PATCH] exit HyperDoc properly" then?
It's not affected by 'wait_for_client_read', because it's MenuServer
instead of SessionManager/ViewportServer.

It looks like the full fix for "sman in pipe" will not come in
before release. I will think of some workaround to build the book.

Also, is the patch 'fix chapter 14 "Browse"' good to go in?

- Qian

On 6/11/24 02:34, Waldek Hebisch wrote:

[Ralf Hemmecke]

> Also, is the patch 'fix chapter 14 "Browse"' good to go in?

+1

Ralf

PS:
Please keep you new-ps-book branch updated (i.e. rebase it).

[Waldek Hebisch]

On Tue, Jun 11, 2024 at 09:12:53PM +0800, Qian Yun wrote:
> I can commit the patch in "[PATCH] exit HyperDoc properly" then?

Yes.

> It's not affected by 'wait_for_client_read', because it's MenuServer
> instead of SessionManager/ViewportServer.
>
> It looks like the full fix for "sman in pipe" will not come in
> before release. I will think of some workaround to build the book.
>
> Also, is the patch 'fix chapter 14 "Browse"' good to go in?

Yes. It looked that changes to book depend on changes to
build process, but once enough changes are commited to build
it it can go in.

--
Waldek Hebisch

[Ralf Hemmecke]

Qian, in commit c9fff275a09df57039b2e9646d30c4fc158744a9
"support build with pdflatex"
the removal of the commented-out-section

%\head{subsection}{Browse Options}{ugBrowseOptions}

in ug14.htex should perhaps also included in this 'fix chapter 14
"Browse"' patch.

That all is anyway independent of the build process.

Ralf

[Qian Yun]

I've updated https://github.com/oldk1331/fricas/commits/new-ps-book/

I included a new workaround in

https://github.com/oldk1331/fricas/commit/d3a777911c5a3de5461acc09379c1c13c350e625

Let me explain:

Now I don't use "sman in pipe" to generate spool files from input files.
Now the spool files are generated via "fricas -nosman" just like before.
So to generate graphic files, I only run "SEGBIND ug00 ug01 ug07 ug08
ug10 ug11" again via sman (and XVFB) to generate those graphic files.

For this workaround, I have to add a new environment "psxtcnooutput"
in htex2input.awk and discard it by "skip-graphics.awk" for normal
spool file generation.

- Qian

[Peter Broadbery]

My plan is to look at this in the next few weeks.  Based on some preliminary work, it looks solvable, as there are similar constructs in other places.  I'd rather not be a bottleneck, so if anyone wants to help with either this or to help make a start on a newer (aldor/spad based) compiler, it would be welcome.

Peter

 

[Ralf Hemmecke]

Hi Qian,

why do you need to rewrite the -eval into a pipe?

@@ -502,6 +500,6 @@ ${GEN_VIEWPORTFILES_PHT}: %.pht: %.ht
${INPUT_EXTRA_INPUT} ${SMAN} \
mobius.VIEW: ${inputsrcdir}/mobius.input
(unset DAASE; FRICAS=${FRICAS}; export FRICAS; \
- FRICAS_INITFILE='' ${XVFB} \
- ${FRICAS}/bin/fricas -noht -noclef -eval ')read $<')
+ echo ')read $<' | FRICAS_INITFILE='' ${XVFB} \
+ ${FRICAS}/bin/fricas -noht -noclef)

On 6/12/24 07:22, Qian Yun wrote:
> I've updated https://github.com/oldk1331/fricas/commits/new-ps-book/

> Now I don't use "sman in pipe" to generate spool files from input files.

That at least compiles fine for me. I am looking at the details.

> Now the spool files are generated via "fricas -nosman" just like before.

That seems to be good.

> So to generate graphic files, I only run "SEGBIND ug00 ug01 ug07 ug08
> ug10 ug11" again via sman (and XVFB) to generate those graphic files.

Yes, as I suggested recently. I am not yet completely done, but I have
the hope that most of the (non-graphic) commands need not be re-run.

> For this workaround, I have to add a new environment "psxtcnooutput"
> in htex2input.awk and discard it by "skip-graphics.awk" for normal
> spool file generation.

My feeling is that this is not necessary. I even do not quite understand
what the meaning of psxtcnooutput is. If I am not completely wrong then
the commands inside \psXtc are anyway not run.

Maybe I go through all the graphics-related stuff and check with the
physical Jenks&Sutor book what should be turned from \xtc to \psXtc. For
example, ")read vectors" or ")read ribbons" or ")read bouquet" produce
no (text) output, only graphics, so they should not be run in
non-graphics pipe, but the commands should be written to the file
ug-pics.input (which you seem to have hand-crafted as far as I currently
understand).

Anyway, I think the idea is good to decouple the pipe that produces the
.tex files from the run that produces the graphics (i.e. the .ps files).
In fact, for HyperDoc, it is also done like this, see the run with
"-paste" in src/doc/Makefile.in.

Thank you for pushing this isssue. And sorry for me being too slow in
following you.

Anyway, eventually I also like to make the .input files go away that
correspond to source code appearing in the .htex files. Probably not
this realease.

Let us now concentrate on generating .ps files and using pdflatex.

Qian, I think your Chapter 14 doc-only changes could go in now. I also
have a few docfixes waiting in my stack. I would like to pile some more
while I go through the docs now and then commit in one patch.

Ralf

[Qian Yun]

On 6/12/24 20:52, Ralf Hemmecke wrote:
> Hi Qian,
>
> why do you need to rewrite the -eval into a pipe?
>
> @@ -502,6 +500,6 @@ ${GEN_VIEWPORTFILES_PHT}: %.pht: %.ht
> ${INPUT_EXTRA_INPUT} ${SMAN} \
>  mobius.VIEW: ${inputsrcdir}/mobius.input
>      (unset DAASE; FRICAS=${FRICAS}; export FRICAS; \
> -        FRICAS_INITFILE='' ${XVFB} \
> -        ${FRICAS}/bin/fricas -noht -noclef -eval ')read $<')
> +        echo ')read $<' | FRICAS_INITFILE='' ${XVFB} \
> +        ${FRICAS}/bin/fricas -noht -noclef)
>

I'm getting errors (and hang) randomly with "sman together with -eval".
I guess you can reproduce this on your side as well.
There seems to be a issue that the "-eval" happens in different frames,
compare:

fricas -eval "x := 1"

fricas -nosman -eval "x := 1"

And check for value of "x" in REPL.

This might indicate another issue that needs investigation later.

>
>> For this workaround, I have to add a new environment "psxtcnooutput"
>> in htex2input.awk and discard it by "skip-graphics.awk" for normal
>> spool file generation.
>
> My feeling is that this is not necessary. I even do not quite understand
> what the meaning of psxtcnooutput is. If I am not completely wrong then
> the commands inside \psXtc are anyway not run.

With current HEAD, commands inside \psXtc are not run.
My patch changes that. But the workaround changes that again,
"skip-graphics.awk" is used to discard commands inside \psXtc to
produce spool files. Then another run without "skip-graphics.awk"
to produce the graphic files.

>
> Qian, I think your Chapter 14 doc-only changes could go in now. I also
> have a few docfixes waiting in my stack. I would like to pile some more
> while I go through the docs now and then commit in one patch.
>
> Ralf
>

On second thought, I think the Chapter 14 change should go just before
the pdflatex change. Because the Chapter 14 change makes a mismatch
between text and image, and pdflatex change fix that.

- Qian

[Ralf Hemmecke]

I am looking at your "generate book with ps image" patch.

>>> For this workaround, I have to add a new environment "psxtcnooutput"
>>> in htex2input.awk and discard it by "skip-graphics.awk" for normal
>>> spool file generation.
>>
>> My feeling is that this is not necessary. I even do not quite
>> understand what the meaning of psxtcnooutput is. If I am not
>> completely wrong then
>> the commands inside \psXtc are anyway not run.
>
> With current HEAD, commands inside \psXtc are not run.
> My patch changes that.   But the workaround changes that again,
> "skip-graphics.awk" is used to discard commands inside \psXtc to
> produce spool files.  Then another run without "skip-graphics.awk"
> to produce the graphic files.

I somehow feel, that this complicates things.

Can we discuss the overall strategy here? The reason is that I would
like to ensure that the HyperDoc generation does not break.

As far as I understand and what I read from Makefile.in is that you
create with htex2input.awk two .input files. One is directly output and
is responsible for creating the graphics files (target
${HTEX_PICS_INPUT}) and the other one pipes that file through
skip-graphics.awk (target ${HTEX_FILES_INPUT}) to generate the .spool file.

Perhaps not a big deal but I looked through the \psXtc commands and to
me it seems that there are only a view cases that commands contained
inside \xtc are needed as a preparation step for \psXtc.
I would actually like to run only those preparation steps together with
the draw commands that produce the .ps files.

Second issue. Also not a big deal, but I was wondering about the newly
introduced src/input/ug-pics.input file. What I learned is that this
file is only there, because every \psXtc now creates a .ps file you have
changed htex2input.awk in such a way that it issues

print "write(%, \"tmp/" $0 "\", \"postscript\"); close(%)"

In particular, the "close(%)" causes the viewport to disappear although
there might be another command afterwards (like a "rotate(vp, 0, -90)"
in chapter 10) that works on the same viewport and produces another .ps
file. Seemingly, our patch puts these extra commands into ug-pics.input.
Yes, that works, but I am not so happy with moving the build logic to
separate handcrafted files.

I would like to suggest the following:

Change htex2input.awk (from "master") in such a way, that is produces
the same tmp/*.input file as in master and additionally a
tmp/pics-*.input file that only contains the commands needed to produce
all the pictures that appear in the respective .htex file.

For that to work we must mark places in the .htex file that have
preparation commands for the \psXtc stuff, like

\xtc{
Extract the space component of \spad{vp}.
}{
\spadcommand{sp := subspace(vp)\bound{d5}\free{d1}}
}

in chapter 10. Simply replacing \xtc by \xtcX and a few changes in
htex2input.awk should be enough.

To overcome the need for the handcrafted ug-pics.input file I suggest
the following.

htex2input.awk has a global variable openViewportExists that is
initialized to 0 at the beginning and set to 1 after any treatment of a
\psXtc command.
Note that in the HyperDoc documentation we find that the places where
viewports are newly created are already properly marked (mod bugs).

https://github.com/fricas/fricas/blob/master/src/doc/ht/HTXAdvPage5.ht#L23

That information can be used. \graphpaste is actually \spadgraph inside
the .htex file and they should only appear inside \psXtc. So, whenever
htex2input.awk sees \spadgraph and openViewportExists==1 then the
the command "close(OPENVIEWPORT)" has to be written to
'tmp/pics-*.input'. Yes, that means that htex2input.awk also must
remember the last created viewport. But that can be done. For
"\spadgraph{vp := draw(...)}" htex2input.awk writes out

OPENVIEWPORT := vp := draw(...)

to the tmp/pics-*.input file. And, of course, htex2input.awk, must contain

END {if (openViewportExists==1) {print "close(OPENVIEWPORT)"}}

Qian, what do you think?

Additionally, the HyperDoc side of the documentation generation only
uses the first two arguments of \psXtc, so we are free to redefine the
meaning of the third one.
That currently always contains something like
\epsffile[0 0 295 295]{ribbon2.ps}
from which the filename is extracted. The stuff inside the square
brackets is not used (as far as I can see). So it would be enough to
simply change that so that \psXtc simply looks like

\psXtc{
some text
}{
\spadgraph{...}
}{
ribbon2.ps
}

However, I am not 100% sure whether we should do that, since the \psXtc
command should then also translate to

\includegraphics{ribbon2.ps}

when it comes to .spool -> .tex translation. I think we do not need it
now, but having only the filename in \psXtc basically prevents us from
giving optional arguments to \includegraphics. OK, we could change
\psXtc{... to \psXtc[options]{ then and move the options to the
\includegraphics.

Opinions?

Ralf

[Qian Yun]

On 6/13/24 18:39, Ralf Hemmecke wrote:
> I am looking at your "generate book with ps image" patch.
>
>>>> For this workaround, I have to add a new environment "psxtcnooutput"
>>>> in htex2input.awk and discard it by "skip-graphics.awk" for normal
>>>> spool file generation.
>>>
>>> My feeling is that this is not necessary. I even do not quite
>>> understand what the meaning of psxtcnooutput is. If I am not
>>> completely wrong then
>>> the commands inside \psXtc are anyway not run.
>>
>> With current HEAD, commands inside \psXtc are not run.
>> My patch changes that.   But the workaround changes that again,
>> "skip-graphics.awk" is used to discard commands inside \psXtc to
>> produce spool files.  Then another run without "skip-graphics.awk"
>> to produce the graphic files.
>
> I somehow feel, that this complicates things.
>
> Can we discuss the overall strategy here? The reason is that I would
> like to ensure that the HyperDoc generation does not break.
>
> As far as I understand and what I read from Makefile.in is that you
> create with htex2input.awk two .input files. One is directly output and
> is responsible for creating the graphics files (target
> ${HTEX_PICS_INPUT}) and the other one pipes that file through
> skip-graphics.awk (target ${HTEX_FILES_INPUT}) to generate the .spool file.

Yes.

> Perhaps not a big deal but I looked through the \psXtc commands and to
> me it seems that there are only a view cases that commands contained
> inside \xtc are needed as a preparation step for \psXtc.
> I would actually like to run only those preparation steps together with
> the draw commands that produce the .ps files.

There are many cases that \psXtc commands need previous \xtc commands.
For example the 2D1VarD.ps, 2DppcE.ps, etc in ug07.

And if you include those commands in \psXtc, then you will not see
their output in latex.

> Second issue. Also not a big deal, but I was wondering about the newly
> introduced src/input/ug-pics.input file. What I learned is that this
> file is only there, because every \psXtc now creates a .ps file you have
> changed htex2input.awk in such a way that it issues
>
>        print "write(%, \"tmp/" $0 "\", \"postscript\"); close(%)"
>
> In particular, the "close(%)" causes the viewport to disappear although
> there might be another command afterwards (like a "rotate(vp, 0, -90)"
> in chapter 10) that works on the same viewport and produces another .ps
> file. Seemingly, our patch puts these extra commands into ug-pics.input.
> Yes, that works, but I am not so happy with moving the build logic to
> separate handcrafted files.

I tried to do something similar to this, but failed.
It is difficult to make hyperdoc and latex happy at the same time.

> I would like to suggest the following:
>
> Change htex2input.awk (from "master") in such a way, that is produces
> the same tmp/*.input file as in master and additionally a
> tmp/pics-*.input file that only contains the commands needed to produce
> all the pictures that appear in the respective .htex file.

The usage of "skip-graphics.awk" is a temporary workaround,
if "sman in pipe" works, this logic can be greatly simplified.

I'm not following your logic here. Seems like it's not solving the
problems in ug10?

- Qian

[Qian Yun]

On 6/13/24 18:39, Ralf Hemmecke wrote:
>

> To overcome the need for the handcrafted ug-pics.input file I suggest
> the following.
>

I have a new idea:

We now have \xtc, \psXtc, \noOutputXtc, \nullXtc:

\xtc: show command, execute command, show result
\psXtc: show command, execute command, don't show result
\noOutputXtc: show command, execute command, don't show result
\nullXtc: show command, don't execute command

I propose to add new type, \shadowXtc:

\shadowXtc: don't show command, execute command, don't show result


This way, I can integrate the additional commands from ug-pics.input
into htex sources.

- Qian

[Ralf Hemmecke]

On 6/14/24 14:39, Qian Yun wrote:
> \xtc:         show command, execute command, show result
> \psXtc:       show command, execute command, don't show result
> \noOutputXtc: show command, execute command, don't show result
> \nullXtc:     show command, don't execute command

Very well described.

> I propose to add new type, \shadowXtc:
>
> \shadowXtc:   don't show command, execute command, don't show result
>
> This way, I can integrate the additional commands from ug-pics.input
> into htex sources.

I do not think that this solves the problem that led to the existence of
ug-pics.input. As far as I understood the problem is that you issue
"close(%)" after creating the .ps file.

Your proposal might remove the need of skip-graphics.awk.

I think we can do better.

\psXtc shows the commands and also writes out the respective command for
execution, but it does that like this
==========
-- \\begin{xtcnooutput}
)if CreateGraphics
spadgraphViewport:=COMMAND
)endif
\\end{xtcnooutput}
==========

Execution is prevented by the fact that CreateGraphics is an unknown
symbol for the )if.

In the run that we create the graphics files, we simply pipe the same
file through fricas, but then with ")assert CreateGraphics" prepended.

I'm currently working on that idea and hope to present a working version
soon.

The basic idea is:

(a) Every \psXtc creates a .ps file.
(b) Every \spadgraph creates a new viewport that is stored in
the special fricas variable "spadGraphViewport" and is closed
right before opening a new one.

\spadgraph only appears inside \psXtc or \nullXtc. We need not care
about \nullXtc.

Ralf

[Ralf Hemmecke]

Hi Qian,

I just see, that in your "support build with pdflatex"
commit, you have basically replaced all the \special commands by
\includehyperdoc. That's good.

I think we should even extend that command like that

\newcommand{\includehyperdoc}[2]{%
\begin{figure}[htbp]
\begin{center}
\includegraphics[scale=0.5]{#1}
\end{center}
\caption{#2}
\end{figure}}

Then we can even more reduce the respective parts in the .htex files to just

\includehyperdoc{FILENAME}{CAPTION}

What do you think?

Ralf

[Qian Yun]

This is a good idea.

BTW, I just updated the branch and pushed a new commit
https://github.com/oldk1331/fricas/commit/5759b0fe43e972b583069de9c81dd1a2fc4a0d91

It didn't add a new xtc environment, instead it add a new command
\shadowspadcommand. It's hard to say if this is better or
the ug-pics.input method is better.

- Qian

[Ralf Hemmecke]

Hi Qian,

I've now created a new-ps-book-rhx branch in my github repo.

https://github.com/hemmecke/fricas/commits/new-ps-book-rhx

I've built that on top of your "generate book with ps image" commit from
your new-ps-book branch.

Since I rewrite quite some stuff from your commit, it is probably better
to squash both commits together.

Let me explain a bit the overall strategy.

In the same spirit I suggested earlier to overcome the "race-problem
with pipe", the .input files that produce the .spool files and
eventually the .tex files are run through FRICASsys (actually interpsys)
directly. No sman involved and no graphics is produced there.

The .input files that contain \spadgraph commands are again run through
fricas/sman in order to produce the .ps pictures.
I have taken care of closing the viewports properly before a new
viewport is opened. (Actually, I even think that is unnecessary, since
they will be closed anyway by )quit at the end. But I realize that only
now when I write this mail and the work to close properly is already done.)

This way, I experience not truncation of files.

The pictures in the book now seem to be a little smaller. So I increased
the size in (htex2input.awk)

print "-- \\includegraphics[height=.24\\textheight]{" $0 "}"

to .25\textheight.

The environment "xtcnooutput" is gone. I use the already existing
"inputonly" environment to include commands that are executed but should
not leave any trace of output in the book.

The skip-graphics.awk script is not needed anymore. The commands from
\psXtc are set inside

)if GenerateGraphics
...
)endif

and the .htex files with \psXtc in them are run twice.
(a) as .input file through FRICASsys to produce .spool
(b) as .input file with

)assert GenerateGraphics

prepended through "fricas -noht -noclef" to produce the pictures.

This avoids the pipe racing problem.

From ug-pics.input only knot3.ps is left, so I renamed and shortened it.
I modified a little so that it looks like the previous black/white
image. I think that looks a bit more fancy than the one in the
Jenks&Sutor book. That's, of course, open for discussion.

Section 7.19 is now with pictures that I have not seen in the FriCAS
1.3.10 version of the book.

Any comments?

I try to adapt your patch to use pdflatex directly.

Ralf

[Qian Yun]

Thanks, preliminary tests are OK, I'll take a deeper look
to the details.

- Qian

[Ralf Hemmecke]

Hi Qian,

I now rebased also your other commits (with the necessary adaptations)
because of my previous commit and added also some modifications to get
rid of tex2tex.awk.

The top commit also removes the space before TeXOutput and the line with
the type.

https://github.com/hemmecke/fricas/commits/new-ps-book-rhx

Now we create book.pdf directly via pdflatex instead of going the
.dvi->.ps->.pdf chain.

Ralf

PS: I am currently experimenting with breqn. It seems that I can get
better line breaks for overfull boxes. At least for some of them.
But that should not hinder a release.

PS2: I'd actually also like to generate the .input files for some of the
pictures from the xmpLines in the .htex files. That, however, seems to
be a task that has to be done with care since interestingly, the code in
the .htex files does not completely agree with what is currently in the
.input files. That will not finish before the release.

[Ralf Hemmecke]

Qian, can you add a line to also install epstopdf into the github CI
environment. Otherwise my new patches break github CI.

2024-06-19T20:37:53.7717736Z /bin/bash: line 1: epstopdf: command not found

Thanks
Ralf

[Qian Yun]

Building hyperdoc should not require epstopdf.

This is caused by "viewports" requires "stamp-book-viewports",
which should not be the case?

- Qian

[Qian Yun]

Thanks for your commit for reducing space around
AlgebraOutput/MessageOutput/TexOutput.

That makes the book shrink from 982 pages to 766 pages!

You should tweak the value a bit, I see AlgebraOutput and
TexOutput overlap a bit.

- Qian

[Ralf Hemmecke]

> Building hyperdoc should not require epstopdf.

Ah, yes, of course.

> This is caused by "viewports" requires "stamp-book-viewports",
> which should not be the case?

No that is OK, but I put too much under stamp-book-viewports.
The call to epstopdf should only occur during 'make book'.
I'll correct that.

BTW, I was hesitating to combine all commits, so that you better see
what I did. I will squash them for inclusion into the official repo.

Maybe commits like this
* generate book with postscript files
* fix chapter 14
* switch to pdflatex
* fix display issue for pascal triangle
* reduces space of output environments (Algebra-, TeX-, MessageOutput)

Ralf

[Ralf Hemmecke]

On 6/20/24 04:12, Qian Yun wrote:
> Thanks for your commit for reducing space around
> AlgebraOutput/MessageOutput/TexOutput.

> You should tweak the value a bit, I see AlgebraOutput and
> TexOutput overlap a bit.

Do you have a particular place where that is apparent?

Or do you rather mean that MessageOutput before TeXOutput
overlap a little? This is what I see, in particular with the output
about "Compiling function" followed by the actual output of the value.

It wouldn't solve that overlapping problem, but I am thinking about
removal of these "Compiling function ..." messages from the book.
They are distracting and irrelevant when reading the book, but on the
other hand they avoid surprises for people who actually type in the
commands and try out the commands from the book.

Opinions?

Ralf

[Qian Yun]

On 6/20/24 15:05, Ralf Hemmecke wrote:
> On 6/20/24 04:12, Qian Yun wrote:
>> Thanks for your commit for reducing space around
>> AlgebraOutput/MessageOutput/TexOutput.
>
>> You should tweak the value a bit, I see AlgebraOutput and
>> TexOutput overlap a bit.
>
> Do you have a particular place where that is apparent?
>
> Or do you rather mean that MessageOutput before TeXOutput
> overlap a little? This is what I see, in particular with the output
> about "Compiling function" followed by the actual output of the value.

Yes, it's MessageOutput before TeXOutput.

> It wouldn't solve that overlapping problem, but I am thinking about
> removal of these "Compiling function ..." messages from the book.
> They are distracting and irrelevant when reading the book, but on the
> other hand they avoid surprises for people who actually type in the
> commands and try out the commands from the book.
>
> Opinions?
>
> Ralf
>

I think we should keep it. In some parts of book they are expected.

> BTW, I was hesitating to combine all commits, so that you better see
> what I did. I will squash them for inclusion into the official repo.

It's better to have separate commits now, and once everything is OK,
you or me can squash relevant commits and push them.

- Qian

[Waldek Hebisch]

IMO we should keep actual output, HyperDoc pages and the FriCAS book
as close to each other as possible. If 'Compiling function' messages
are deemed distractiong, then we could supress them in default output.
OTOH they give information than sometimes is useful.

Also, more general remark: whitespace is cheap and can significantly
improve readability. So, remove whitespace when it is trurely
excessive, but do not aim to reduce book length in this way.
--
Waldek Hebisch

[Ralf Hemmecke]

I've pushed another update to

https://github.com/hemmecke/fricas/commits/new-ps-book-rhx

What actually still bothers me a bit is that breqn does not break some
output. I seem to have found the reason. Obviously the mleftright
package does not play nicely with breqn. I basically introduces \mleft
and \mright for the \PAREN macro

https://github.com/hemmecke/fricas/blob/master/src/doc/fricasmath.sty#L57

in order to over come an extra space between sin and ( in the usual
"\sin\left(x\right)". "\sin\mleft(x\mright)" avoids such a space and is
more or less equivalent to "\sin\!\left(x\right)".

Maybe I must rewrite fmtlatex.spad a little to overcome that problem and
then remove the need for the mleftright package.

On 6/20/24 18:06, Waldek Hebisch wrote:
>> It wouldn't solve that overlapping problem, but I am thinking about removal
>> of these "Compiling function ..." messages from the book.
>> They are distracting and irrelevant when reading the book, but on the other
>> hand they avoid surprises for people who actually type in the commands and
>> try out the commands from the book.
>
> IMO we should keep actual output, HyperDoc pages and the FriCAS book
> as close to each other as possible. If 'Compiling function' messages
> are deemed distractiong, then we could supress them in default output.
> OTOH they give information than sometimes is useful.

OK, then we leave that in the book.

> Also, more general remark: whitespace is cheap and can significantly
> improve readability. So, remove whitespace when it is trurely
> excessive, but do not aim to reduce book length in this way.

My intention was not to remove the number of pages, but to show the
output of the pascal triangle nicely (which comes from fricas as
separate chunks and therefore in separate latex environments).
Additionally, I wanted to put the output of a command closer together.
There is no need to put a gap between the value of a command and its
type. The different types of the output are anyway separated by color.
I left a little space between input an output though.

Ralf

[Ralf Hemmecke]

On 6/21/24 00:29, Ralf Hemmecke wrote:
I've pushed yet another update to

https://github.com/hemmecke/fricas/commits/new-ps-book-rhx

The line break problem in connection with parentheses is now gone.

There are still a few places where lines do not break.

One case is too long integers of floats. Maybe that can be solved, but
certainly not before the release.

Another case is fractions with numerators of denominators that do not
fit on one line. Fortunately, all these cases are just with \frac at the
top-level. If I find time, maybe I can transform this case into
typesetting the numberator and denominator separately. I guess that is
not totally easy to get it right.

I also found cases in connection with GroupRepresentations and
Permutations. I'm not sure what to do there. I have to look more closely.

I just realize that a rewriting of section 9.48.1 "Differential
operators with series coefficients" differs completely from Chapter
9.35.1 in the Jenks&Sutor book. In particular, I think there are too
many "Compiling ..." lines. This does not look nice in a book, in my
opinion. Can that perhaps be rewritten with macros instead of functions?
Suggestion attached.

I think after Qian's patch for showing the print output in the book

fix display issue of "output" in book section 6.19

, it also has an effect on Chapter 13.18.8.

Do we now want the full output of 13.13.8 "Example Queries" (see the
long output of function signatures at the end of the chapter)?

Maybe we should change to query into something with less output. The
output for the book is rather uninteresting. The idea of *how* one can
get this information can also be conveyed by a less verbose output.
Any suggestions?

I think, if Qian does not find problems, I can squash the commits and
push them upstream.

Waldek, do you have any preference of what should be kept separate?

Ralf

[Waldek Hebisch]

On Sat, Jun 22, 2024 at 11:01:11PM +0200, Ralf Hemmecke wrote:
> On 6/21/24 00:29, Ralf Hemmecke wrote:
> I've pushed yet another update to
>
> https://github.com/hemmecke/fricas/commits/new-ps-book-rhx
>
> The line break problem in connection with parentheses is now gone.
>
> There are still a few places where lines do not break.

You mean: places where lines do not break now but did in previous
version? Or places where problem already appeared in earlier
versions? If problem is old I would prefer to limit time spent
on it. AFAICS current book in a improvement, it does not make
sense to delay much to solve all problems. And in particular
rushed solutions tend to have their own problems.

> One case is too long integers of floats. Maybe that can be solved, but
> certainly not before the release.
>
> Another case is fractions with numerators of denominators that do not fit on
> one line. Fortunately, all these cases are just with \frac at the top-level.
> If I find time, maybe I can transform this case into typesetting the
> numberator and denominator separately. I guess that is not totally easy to
> get it right.
>
> I also found cases in connection with GroupRepresentations and Permutations.
> I'm not sure what to do there. I have to look more closely.
>
> I just realize that a rewriting of section 9.48.1 "Differential operators
> with series coefficients" differs completely from Chapter 9.35.1 in the
> Jenks&Sutor book. In particular, I think there are too many "Compiling ..."
> lines. This does not look nice in a book, in my opinion. Can that perhaps be
> rewritten with macros instead of functions?
> Suggestion attached.

So you are replacing seqence of definitions by single function. At
first glance that looks good (I did not check how it looks in generated
book).

> I think after Qian's patch for showing the print output in the book
>
> fix display issue of "output" in book section 6.19
>
> , it also has an effect on Chapter 13.18.8.
>
> Do we now want the full output of 13.13.8 "Example Queries" (see the long
> output of function signatures at the end of the chapter)?
>
> Maybe we should change to query into something with less output. The output
> for the book is rather uninteresting. The idea of *how* one can get this
> information can also be conveyed by a less verbose output.
> Any suggestions?

You mean output of 'fullDisplay(fm-%)'? IMO it is reasonable to
completely skip it, it just essentially repeats previous points.
Or repleace it by 'fullDisplay(fm-%, 1, 5)' with text changed
to 'Display first 5 operations ...'.

--
Waldek Hebisch

[Ralf Hemmecke]

On 6/22/24 23:54, Waldek Hebisch wrote:
>> There are still a few places where lines do not break.
>
> You mean: places where lines do not break now but did in previous
> version?

Of course not. Now more places are resolved. And only about 10 or so are
left. These places are connected to long integers/floats most other
places are from code that is not in the orginal Jenks & Sutor book
(except the very first in Chapter 0.1).

> If problem is old I would prefer to limit time spent on it. AFAICS
> current book in a improvement, it does not make sense to delay much
> to solve all problems. And in particular rushed solutions tend to
> have their own problems.

I agree. for me it is good enough and better than before. No reason to
delay any further if the code on my new-ps-book-rhx branch is committed.
Qian's work on generating the .ps file is really an improvement.

> So you are replacing seqence of definitions by single function. At
> first glance that looks good (I did not check how it looks in
> generated book).

The output is as before, but the many red lines on pages 593pp
are no longer there, because no recompilation happens.

https://fricas.github.io/book.pdf

So, agreed to include that change to section 9.48.1?

> You mean output of 'fullDisplay(fm-%)'?

Yes.

> IMO it is reasonable to completely skip it, it just essentially
> repeats previous points. Or repleace it by 'fullDisplay(fm-%, 1, 5)'
> with text changed to 'Display first 5 operations ...'.

Done in commit P below.

I have now reordered the commits so that commits-to-be-squashed
are together.

https://github.com/hemmecke/fricas/commits/new-ps-book-rhx

R remove unused \hfil from util.ht
Q docfix
P shorten too verbose output in section "ugDomainsExamples"
O rewrite LODO section of the book
N fix for better line breaking with breqn
M update list of required latex packages
L update list of required latex packages
K adjust/reduce space of fricas output
J reduce vertical space before TexOutput and \formatResultType
I AlgebraOutput and MessageOutput without space above and below
H fix display issue of "output" in book section 6.19
G split target stamp-book-viewports
F fix use of pdflatex
E support build with pdflatex
D fix chapter 14 "Browse"
C generate book.pdf with postscript files
B generate book with ps image
A docfix for HT ug00

I suggest to squash B-C, E-F-G, I-J-K, L-M, and maybe
O-P under "revise book sections"
or O-P-Q-R under "various doc improvements" (actually also A can go there)

What is the preference?

Is it necessary to add a ChangeLog entry for each of the separate
commits (after squashing)? I certainly do one where there is work of
both Qian and me.

Ralf

[Qian Yun]

Hi Ralf,

In your commit A, I think it's better to use "b*x" instead of
"bx" in the first diff block, and there's a missing right
parenthesis in "log(cot(x)".

Commit G should be merged with commit B.
But I still think target "viewports" should not depend on
target "stamp-book-viewports".

I'm doing final testings, if everything is OK, I'll
commit B/C/G, D, E/F, H/I with Changelogs and attributions to you.
You can handle the rest commits as you see fit.

(I've included one small change that move "subspace(vp)" of section
10.1 to psXtc.

I also added target "${PS_FILES_PS}" to "stamp-book-pics" to make
commit B build.
)

I've updated the squashed and rebased branch to
https://github.com/oldk1331/fricas/commits/new-ps-book-final/

If there are no further comments, I'd like to merge those 4
long overdue commits.

- Qian

[Waldek Hebisch]

On Sun, Jun 23, 2024 at 01:15:02PM +0200, Ralf Hemmecke wrote:
> On 6/22/24 23:54, Waldek Hebisch wrote:
>
> > So you are replacing seqence of definitions by single function. At
> > first glance that looks good (I did not check how it looks in generated
> > book).
>
> The output is as before, but the many red lines on pages 593pp
> are no longer there, because no recompilation happens.
>
> https://fricas.github.io/book.pdf
>
> So, agreed to include that change to section 9.48.1?

Yes.

I have small comments to other parts. On page 1 in section
Authors we should add Qian Yun. The 'update list of required latex
packages' commit leaves 'mleftright' in INSTALL (IIUC it is no
longer needed).

I think that O-P-Q-R-A as a single commit is good. Definitely
E-F-G in a single part and B-C in its part. And I would bundle
L-M with change that made 'mleftright' unnecessary.

> Is it necessary to add a ChangeLog entry for each of the separate commits
> (after squashing)? I certainly do one where there is work of both Qian and
> me.

No. I normally do not add ChangeLog entries for pure documentation/comment
changes. So add entries when you feel they are needed.


--
Waldek Hebisch

[Ralf Hemmecke]

On 6/23/24 17:45, Waldek Hebisch wrote:
> I have small comments to other parts. On page 1 in section
> Authors we should add Qian Yun.

OK, that is new commit S then.

> The 'update list of required latex
> packages' commit leaves 'mleftright' in INSTALL (IIUC it is no
> longer needed).

Thank you for looking so closely. I'll correct patch M.

On 6/23/24 16:01, Qian Yun wrote:
> In your commit A, I think it's better to use "b*x" instead of
> "bx" in the first diff block, and there's a missing right
> parenthesis in "log(cot(x)".

Thank's Qian. I'll correct patch A.

> Commit G should be merged with commit B.

I think you did it correctly.

> But I still think target "viewports" should not depend on
> target "stamp-book-viewports"

Well, maybe a good point. The initial reason why I included it into
viewport was that the target "viewport" exists, because if xvfb is not
existing, the pictures are not created for HyperDoc.
You are in a sense right that target doc in src/doc/Makefile is not
build by a top-level make, i.e. the book is only built on explicit call
to make book in src/doc. It should be come as no surprise if there will
be many windows popping up during "make doc" in case xvfb is not
available. Now, thinking about this issue, maybe you are right and
stamp-book-viewports can be removed from viewports. Please check whether
the building of the book viewports is triggered by "make doc". The
pictures (even in a parallel) build must be available before latex is
started. I faintly remember that some pictures weren't yet available
during the latex run. So make-book must be dependend on
stamp-book-viewports.

> I'm doing final testings, if everything is OK, I'll
> commit B/C/G, D, E/F, H/I with Changelogs and attributions to you.
> You can handle the rest commits as you see fit.

Basically OK, but I still have a few comments to B/C/G.

> (I've included one small change that move "subspace(vp)" of section
> 10.1 to psXtc.

Sorry, I feel that this is not correct. Execute the commands from the
generated .input file. \psXtc is supposed to produce a .ps file.
You have not given a name for such a file in the third argument of
\psXtc. Does the generated write(...) command connected to \psXtc
actually work without error?
Fortunately, it is \spadcommand{sp := subspace(vp)\bound{d5}\free{d1}}
and not \spadgraph, because otherwise the viewport vp would have been
closed before that subspace(vp) command is executed, which would produce
an error, I guess.

> I also added target "${PS_FILES_PS}" to "stamp-book-pics" to make
> commit B build.

Seem to look OK.

> I've updated the squashed and rebased branch to
> https://github.com/oldk1331/fricas/commits/new-ps-book-final/
>
> If there are no further comments, I'd like to merge those 4
> long overdue commits.

I am still rechecking your squashed stuff, but I am too tired right now.
Hopefully I can manage tomorrow night.

Ralf

[Ralf Hemmecke]

Hi Qian,

> On 6/23/24 16:01, Qian Yun wrote:
> > Commit G should be merged with commit B.
>
> I think you did it correctly.
>
> > But I still think target "viewports" should not depend on
> > target "stamp-book-viewports"

OK, let's try without that dependency. The book-pdf target depends on
the pictures, so everything is fine.

> > I'm doing final testings, if everything is OK, I'll
> > commit B/C/G, D, E/F, H/I with Changelogs and attributions to you.
> > You can handle the rest commits as you see fit.

OK under the conditions below.

> > (I've included one small change that move "subspace(vp)" of section
> > 10.1 to psXtc.

You should not include this part as I mentioned in my last mail.

> > I also added target "${PS_FILES_PS}" to "stamp-book-pics" to make
> > commit B build.

OK.

> > I've updated the squashed and rebased branch to
> > https://github.com/oldk1331/fricas/commits/new-ps-book-final/

You should change "tmp/vp-input" in Makefile.in back to "tmp/vp-*.input".

-# - Run the tmp/vp-input files through fricas (target
${HTEX_VIEWPORTS_STAMP})
+# - Run the tmp/vp-*.input files through fricas (target
${HTEX_VIEWPORTS_STAMP})

> > If there are no further comments, I'd like to merge those 4
> > long overdue commits.

You have my OK.

Thank you, Qian.

I'll then prepare the remaining changes.

Ralf

[Qian Yun]

On 6/25/24 16:44, Ralf Hemmecke wrote:
> Hi Qian,
>
>> On 6/23/24 16:01, Qian Yun wrote:
>>  > Commit G should be merged with commit B.
>>
>> I think you did it correctly.
>>
>>  > But I still think target "viewports" should not depend on
>>  > target "stamp-book-viewports"
>
> OK, let's try without that dependency. The book-pdf target depends on
> the pictures, so everything is fine.
>
>>  > I'm doing final testings, if everything is OK, I'll
>>  > commit B/C/G, D, E/F, H/I with Changelogs and attributions to you.
>>  > You can handle the rest commits as you see fit.
>
> OK under the conditions below.
>
>>  > (I've included one small change that move "subspace(vp)" of section
>>  > 10.1 to psXtc.
>
> You should not include this part as I mentioned in my last mail.

Then "vp" is not defined, causing error messages in the book, you can
verify that.

Also I checked the generated tmp/vp-ug10.input, since the third argument
to \psXtc is empty, the htex2input.awk will not generate faulty "write"
commands.

Thanks for your review. After some final testings, I'll do the commit
later.

- Qian

[Ralf Hemmecke]

Hi Qian,

>>>  > (I've included one small change that move "subspace(vp)" of section
>>>  > 10.1 to psXtc.
>>
>> You should not include this part as I mentioned in my last mail.
>
> Then "vp" is not defined, causing error messages in the book, you can
> verify that.

sorry, it took a while until I understood your remark.

What you actually mean is that vp is not defined when the .input file is
given to FRICASsys to produce the .spool file. I was wrongly only
considering the graphics generation side.

Yes, for that you are right. And that should not happen, so \psXtc would
be the right thing to avoid that.

I think all is good for now, but looking at the generated .input file, I
am actually not completely happy.

For example, there is a rotate(vp, 0, -90) command in ug10.htex.
Currently, we generate that command as

-- \begin{inputonly}
rotate(vp, 0, -90)
-- \end{inputonly}

so it is executed, but the output is ignored.

Better would have been

-- \begin{inputonly}
)if CreateGraphics
rotate(vp, 0, -90)
)endif
-- \end{inputonly}

as in the other \psXtc commands.

The reason for this mild problem is that I thought that only the
\spadgraph commands are graphical stuff that do not need execution in
the run to generated the output for the book, i.e., the spool file.
Now I believe that, it is actually any command inside \psXtc that does
not need to be executed in a non-graphical run. So you might
change in htex2input.awk

if (xtcname=="psXtc") {
if (posspadgraph>0) {maybeCloseViewport()}
print "-- \\begin{inputonly}"
if (posspadgraph>0) {
spadgraph=1
print ")if CreateGraphics"
print "spadgraphViewport:=" $0
print ")endif"
} else {
print $0
}
print "-- \\end{inputonly}"
}

into

if (xtcname=="psXtc") {
if (posspadgraph>0) {maybeCloseViewport()}
print "-- \\begin{inputonly}"
print ")if CreateGraphics"
if (posspadgraph>0) {
spadgraph=1
print "spadgraphViewport:=" $0
} else {
print $0
}
print ")endif"
print "-- \\end{inputonly}"
}


There are, in fact, only 4 cases where that matters.

find . -type f -name '*.htex' -exec grep --color -nH -e
'\\psXtc\|\\spadgraph\|\\spadcommand' \{\} +|grep -A1 '\\psXtc'|grep -v
'\\psXtc'|grep -v '\\spadgraph'
./htex/ug10.htex:134:\spadcommand{rotate(vp, 0, -90)\bound{d3}\free{d1}}
--
./htex/ug10.htex:194:\spadcommand{drawStyle(vp,"shade");outlineRender(vp,"on")\bound{d10}\free{d6}}
./htex/ug10.htex:201:\spadcommand{rotate(vp,20,-60);
showRegion(vp,"on")\bound{d11}\free{d10}}
--
--
./htex/ug10.htex:488:\spadcommand{rotate(vp,200,-60)\bound{v5}\free{v4}}

So it looks like the above modification should work. (And they do on my
machine.) You can include the attached patch to your "generate book with
ps image" patch.

> Also I checked the generated tmp/vp-ug10.input, since the third argument
> to \psXtc is empty, the htex2input.awk will not generate faulty "write"
> commands.

Well, it is true that there will be no write command generated, but the
reason is not that there is no filename, but rather that inside \psXtc
is \spadcommand and not \spadgraph. So all is fine then.

> Thanks for your review.  After some final testings, I'll do the commit
> later.

Maybe you include the attached patch in your first commit.

Ralf

[Waldek Hebisch]

On Tue, Jun 25, 2024 at 05:48:11PM +0800, Qian Yun wrote:
>
> Thanks for your review. After some final testings, I'll do the commit
> later.

I get the following message:

warning: CRLF will be replaced by LF in src/doc/png/2Dctrl.png.

and several similar messages for other .png files.

Runinig 'git check-attr' gives:

git check-attr -a src/doc/png/2Dctrl.png
src/doc/png/2Dctrl.png: text: auto
src/doc/png/2Dctrl.png: eol: lf
src/doc/png/2Dctrl.png: linguist-detectable: unset

I am affraid that 'text: auto' is to blame, the files probably
should be marked as binary.

--
Waldek Hebisch

[Ralf Hemmecke]

On 6/28/24 20:40, Waldek Hebisch wrote:
> I get the following message:
>
> warning: CRLF will be replaced by LF in src/doc/png/2Dctrl.png.
>
> and several similar messages for other .png files.

When do you get this and on what type of machine.

> I am affraid that 'text: auto' is to blame, the files probably
> should be marked as binary.

Oh, I must have slept. Qian, can you explain why you think that the line

* text=auto eol=lf

is necessary in .gitattributes? AFAIU, it says that every file is to be
considered as a text file. Why?
Secondly, why would you want to force just LF on Windows?

I think all that got unnoticed, because we had now binary files before png.

Perhaps now we additionally need a line

*.png -text

Honestly, I am rather for removing "* text=auto eol=lf", since all the
people with write access to the official repo anyway work on *nix, no?

Ralf

[Waldek Hebisch]

On Fri, Jun 28, 2024 at 11:55:37PM +0200, Ralf Hemmecke wrote:
> On 6/28/24 20:40, Waldek Hebisch wrote:
> > I get the following message:
> >
> > warning: CRLF will be replaced by LF in src/doc/png/2Dctrl.png.
> >
> > and several similar messages for other .png files.
>
> When do you get this and on what type of machine.

That is on my home machine. But I have seen in the past similar messages
for other files on different machines, so I do not think this is
important.

> > I am affraid that 'text: auto' is to blame, the files probably
> > should be marked as binary.
>
> Oh, I must have slept. Qian, can you explain why you think that the line
>
> * text=auto eol=lf
>
> is necessary in .gitattributes? AFAIU, it says that every file is to be
> considered as a text file. Why?
> Secondly, why would you want to force just LF on Windows?

Well, AFAICS this is correct setting for our normal files and we
had equvalent setting in SVN for many years. We just need to
correctly mark true binary files.

> I think all that got unnoticed, because we had now binary files before png.

We had binary files in the past. Quite possible that we got rid of
them before Qian commited png-s.

> Perhaps now we additionally need a line
>
> *.png -text
>
> Honestly, I am rather for removing "* text=auto eol=lf", since all the
> people with write access to the official repo anyway work on *nix, no?

ATM Windows exists and Mac OS also have strange handling of ends of
lines. I would be unwise to change setting just looking at
current commiters.

--
Waldek Hebisch

[Waldek Hebisch]

On Fri, Jun 28, 2024 at 11:55:37PM +0200, Ralf Hemmecke wrote:

> Perhaps now we additionally need a line
>
> *.png -text

Yes, I think so. Adding it removes the warning message.

--
Waldek Hebisch

[Qian Yun]

On 6/29/24 05:55, Ralf Hemmecke wrote:
>
> Oh, I must have slept. Qian, can you explain why you think that the line
>
> * text=auto eol=lf
>
> is necessary in .gitattributes? AFAIU, it says that every file is to be
> considered as a text file. Why?

From "man gitattributes": When text is set to "auto", Git decides by
itself whether the file is text or binary.

When I clone fricas git repo on my machine, I don't have this warning.

Maybe Waldek's git version is old?

> Secondly, why would you want to force just LF on Windows?

We had problems when compiling Boot files with CRLF endings.

Also it would be messy for a Windows developer to commit without
this setting.

> I think all that got unnoticed, because we had now binary files before png.
>
> Perhaps now we additionally need a line
>
> *.png        -text
>

To Waldek, since I can't test this on my side, if this works for you,
you can commit this change.

- Qian