\spad HT command

[Grégory Vanuxem]

Hello,

During FriCAS compilation of the Algebra in a terminal we can see
"Unexpected HT command: \spad" displayed numerous times. Any idea why
this command
seems no longer supported. Is this a bug or a step in the process of
removing it? I may be wrong but I would not be surprised if it was
used to display the brace enclosed content in italic. Example:

--->-->OpenMathPackage((OMsupportsSymbol? ((Boolean) (String)
(String)))): Unexpected HT command: \spad
--->-->OpenMathPackage((OMsupportsSymbol? ((Boolean) (String)
(String)))): Unexpected HT command: \spad
--->-->OpenMathPackage((OMsupportsSymbol? ((Boolean) (String)
(String)))): Unexpected HT command: \spad

Regards,

- Greg

PS: I can fill the content of this mail as an issue on GitHub if necessary.

[Waldek Hebisch]

On Tue, Dec 26, 2023 at 08:14:28AM +0100, Grégory Vanuxem wrote:
> Hello,
>
> During FriCAS compilation of the Algebra in a terminal we can see
> "Unexpected HT command: \spad" displayed numerous times. Any idea why
> this command
> seems no longer supported. Is this a bug or a step in the process of
> removing it? I may be wrong but I would not be surprised if it was
> used to display the brace enclosed content in italic. Example:
>
> --->-->OpenMathPackage((OMsupportsSymbol? ((Boolean) (String)
> (String)))): Unexpected HT command: \spad
> --->-->OpenMathPackage((OMsupportsSymbol? ((Boolean) (String)
> (String)))): Unexpected HT command: \spad
> --->-->OpenMathPackage((OMsupportsSymbol? ((Boolean) (String)
> (String)))): Unexpected HT command: \spad

This is recent regression caused by commit "Use '|$spadroot|' instead
of '(|getEnv| "FRICAS")'" (4d47c00545dc9aa82ae866adb46332a654348bc4).

We need to make sure that FriCAS can find 'util.ht' during build.

--
Waldek Hebisch

[Grégory Vanuxem]

Ah ok, thanks. Qian will look at this I think.

--
You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/ZYrZ8z1P--GWGRUb%40fricas.org.

[Qian Yun]

A quick and dirty fix is:

=====
diff --git a/src/doc/Makefile.in b/src/doc/Makefile.in
index e968affd..542f6cba 100644
--- a/src/doc/Makefile.in
+++ b/src/doc/Makefile.in
@@ -255,7 +255,7 @@ GEN_HTFILES = xmpexp coverex gloss
# The reason for this delayed execution is that generation of the .xpm
# files in *.VIEW/ without the xvfb-run program will pop up image
# windows in X and will make working during compilation impossible.
-# The intro.htex and util.ht files are not relevant for image generation.
+# The util.ht file is used during generation of interpsys.
VIEWPORTFILES = coverex \
$(shell grep -lRe 'graphpaste{\|spadgraph{' ${srcdir}/ht/*
${srcdir}/htex/* \
| sed 's,.*/,,;s,\..*,,' \
diff --git a/src/interp/htcheck.boot b/src/interp/htcheck.boot
index e3f78805..bc2d5596 100644
--- a/src/interp/htcheck.boot
+++ b/src/interp/htcheck.boot
@@ -80,7 +80,8 @@ $primitiveHtCommands := '(

buildHtMacroTable() ==
$htMacroTable := MAKE_HASHTABLE('EQUAL)
- fn := CONCAT($spadroot, '"/share/hypertex/pages/util.ht")
+ -- this is called once by bootsys to generate interpsys
+ fn := CONCAT(getEnv '"FRICAS", '"/share/hypertex/pages/util.ht")
if PROBE_-FILE(fn) then
instream := MAKE_INSTREAM(fn)
while not EOFP instream repeat
=====

The reason is that '|$spadroot|' is not correct when dumping
interpsys by bootsys.

Let's see if there's a better way.

- Qian

[Qian Yun]

I believe this patch is the correct way to fix this patch.

- Qian

diff --git a/src/doc/Makefile.in b/src/doc/Makefile.in
index e968affd..542f6cba 100644
--- a/src/doc/Makefile.in
+++ b/src/doc/Makefile.in
@@ -255,7 +255,7 @@ GEN_HTFILES = xmpexp coverex gloss
# The reason for this delayed execution is that generation of the .xpm
# files in *.VIEW/ without the xvfb-run program will pop up image
# windows in X and will make working during compilation impossible.
-# The intro.htex and util.ht files are not relevant for image generation.
+# The util.ht file is used during generation of interpsys.
VIEWPORTFILES = coverex \
$(shell grep -lRe 'graphpaste{\|spadgraph{' ${srcdir}/ht/*
${srcdir}/htex/* \
| sed 's,.*/,,;s,\..*,,' \

diff --git a/src/interp/Makefile.in b/src/interp/Makefile.in
index 0576330a..66e289a3 100644
--- a/src/interp/Makefile.in
+++ b/src/interp/Makefile.in
@@ -132,6 +132,7 @@ makeint.lisp: ../boot/lobj_lst
' (quote (' >> makeint.lisp
cat ../boot/lobj_lst >> makeint.lisp
echo '))))' >> makeint.lisp
+ echo '(initroot)' >> makeint.lisp
echo '(build-interpsys' \
'(quote ($(patsubst %, "%", ${ALL_OBJS}))))' >> makeint.lisp
@ echo '(in-package "BOOT")' >> makeint.lisp

[Ralf Hemmecke]

Hi Qian,

I do not know how relevant that is, but util.ht is pretty special and
has a separate entry "util-ht" in src/doc/Makefile.in.

https://github.com/fricas/fricas/blob/master/src/doc/Makefile.in#L328

It will be generated before the compilation process comes src/doc for
producing the documentation.

Instead of your change to the documentation I would like to have this
somehow a little more precise. Maybe like this:

===========
# The intro.htex and util.ht files are not directly relevant for image
# generation, however, util.ht is used during generation of interpsys.
===========

Unfortunately, I have not followed the move from $FRICAS to $spadroot,
so I cannot say anything about your change to src/interp/htcheck.boot.

Can you remind me what the relation between the FRICAS variable and
spadroot actually is? If FRICAS is set at the start of fricas will then
spadroot take this value or what is the actual logic?

Ralf

[Qian Yun]

On 12/27/23 19:05, Ralf Hemmecke wrote:
> Hi Qian,
>
> I do not know how relevant that is, but util.ht is pretty special and
> has a separate entry "util-ht" in src/doc/Makefile.in.
>
> https://github.com/fricas/fricas/blob/master/src/doc/Makefile.in#L328
>
> It will be generated before the compilation process comes src/doc for
> producing the documentation.

But it looks like it is a directly copy of src/doc/ht/util.ht,
not generated.

> Instead of your change to the documentation I would like to have this
> somehow a little more precise. Maybe like this:
>
> ===========
> # The intro.htex and util.ht files are not directly relevant for image
> # generation, however, util.ht is used during generation of interpsys.
> ===========

Does "intro.htex" still exist? If so, where?

> Unfortunately, I have not followed the move from $FRICAS to $spadroot,
> so I cannot say anything about your change to src/interp/htcheck.boot.
>
> Can you remind me what the relation between the FRICAS variable and
> spadroot actually is? If FRICAS is set at the start of fricas will then
> spadroot take this value or what is the actual logic?
>
> Ralf
>

When function "initroot" is called, it will take the value of
environment variable "FRICAS" (if it exists) and assign to Lisp
variable |$spadroot|.

- Qian

[Grégory Vanuxem]

Hello Qian,

I don't know if it is the correct way but it fixes this issue. To use
my previous example:

========================
)compile OMPKG.spad

Compiling FriCAS source code from file
/home/greg/Git/jlfricas/src/algebra/OMPKG.spad using old system
compiler.
OMPKG abbreviates package OpenMathPackage
------------------------------------------------------------------------
initializing NRLIB OMPKG for OpenMathPackage
compiling into NRLIB OMPKG

;;; *** |OpenMathPackage| REDEFINED

;;; *** |OpenMathPackage| REDEFINED
Time: 0 SEC.


Cumulative Statistics for Constructor OpenMathPackage
Time: 0 seconds

finalizing NRLIB OMPKG
; compiling file
"/home/greg/Git/jlfricas/src/algebra/OMPKG.NRLIB/OMPKG.lsp" (written
27 DEC 2023 09:34:40 AM):

; wrote /home/greg/Git/jlfricas/src/algebra/OMPKG.NRLIB/OMPKG.fasl
; compilation finished in 0:00:00.000

========================

Thanks, this was really annoying, especially when you need to look at
the output during compilation, say, for personal stuff.

Regards,

- Greg

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/de425311-7075-48e2-b52f-38d310fbdc7a%40gmail.com.

[Ralf Hemmecke]

Hi Qian,

>> https://github.com/fricas/fricas/blob/master/src/doc/Makefile.in#L328
>>
>> It will be generated before the compilation process comes src/doc for
>> producing the documentation.
>
> But it looks like it is a directly copy of src/doc/ht/util.ht,
> not generated.

Oh yes, you are right. I had to look it up myself. I believe the point
was that util.ht is not only needed in src/doc, but elsewhere.

Maybe src/interp/htcheck.boot is this "elsewhere". And it seems to be
the place to teach hyperdoc about the macros. As far as I can say,
util.ht is for HyperDoc what is a .sty file for LaTeX.

In fact, I faintly remember that I wanted to move util.ht to some other
place, but kept it under src/doc because it will be installed along with
all the other .ht files. If I am not completely wrong then it is not
used at all during compilation in src/doc, but rather only during "make
install".

The reason why I believe this is that src/doc/Makefile.in just produces
book.pdf from the .htex files (and the generated images) and the .html
files (from the .rst files). There will be some .ht files produced from
the .htex files, but I cannot remember that I ever actively *use* .ht
files other than for copying them to the respective installation directory.

>> Instead of your change to the documentation I would like to have this
>> somehow a little more precise. Maybe like this:
>>
>> ===========
>> # The intro.htex and util.ht files are not directly relevant for image
>> # generation, however, util.ht is used during generation of interpsys.
>> ===========
>
> Does "intro.htex" still exist? If so, where?

Oh, good catch. "intro.tex" existed on the NAGcdrom. I do not remember
why I have left that in the comments, but the content of that file now
essentially lives in ug00.htex. There should be no reason to mention
ug00.htex. I think you can simply remove the reference to intro.htex.

I definitely did not mean tecintro.tex, because that never existed in
.htex format.

https://github.com/fricas/fricas/blob/master/src/doc/book.tex#L72

> When function "initroot" is called, it will take the value of
> environment variable "FRICAS" (if it exists) and assign to Lisp
> variable |$spadroot|.

Yes, OK. The idea was to get rid of the FRICAS variable in the internals
of FRICASsys and rather rely on spadroot, right? I actually never
understood why this would matter whether the internal variable is called
FRICAS or spadroot. Anyway, then your first patch

- fn := CONCAT($spadroot, '"/share/hypertex/pages/util.ht")
+ -- this is called once by bootsys to generate interpsys
+ fn := CONCAT(getEnv '"FRICAS", '"/share/hypertex/pages/util.ht")

looked like a bad thing, since it intruces FRICAS again.

The second patch

+ echo '(initroot)' >> makeint.lisp

looks better, but I do not dare to say anything about whether it solves
the problem in the right way.

Ralf

[Waldek Hebisch]

Looks OK, please commit.

--
Waldek Hebisch

[Waldek Hebisch]

On Wed, Dec 27, 2023 at 12:05:26PM +0100, Ralf Hemmecke wrote:
> Hi Qian,

<snip>

> Instead of your change to the documentation I would like to have this
> somehow a little more precise. Maybe like this:
>
> ===========
> # The intro.htex and util.ht files are not directly relevant for image
> # generation, however, util.ht is used during generation of interpsys.
> ===========

'util.ht' is relevant for image generation: it contains macros used
by HyperDoc. No comment is better than wrong comment, so it is
really best to remove this part.

--
Waldek Hebisch

[Ralf Hemmecke]

Are you saying that

================
${GEN_VIEWPORTFILES_PHT}: %.pht: %.ht ${INPUT_EXTRA_INPUT} ${SMAN} \
$(fricas_target_bindir)/hypertex$(EXEEXT) stamp-initial-ht-db
echo $<
unset DAASE; HTPATH=.; export HTPATH; \
FRICAS_INITFILE='' ${XVFB} ${SMAN} -noihere -paste $<

mobius.VIEW: ${inputsrcdir}/mobius.input
(unset DAASE; FRICAS=${FRICAS}; export FRICAS; \
FRICAS_INITFILE='' ${XVFB} \
${FRICAS}/bin/fricas -noht -noclef -eval ')read $<')
===============

in src/doc/Makefile.in needs src/doc/ht/util.ht?

Ralf

[Waldek Hebisch]

Well, first rule needs 'util.ht'. I did not check which one.
'mobius.VIEW' can be generated without using 'util.ht'.

--
Waldek Hebisch

[Ralf Hemmecke]

>> Are you saying that
>>
>> ================
>> ${GEN_VIEWPORTFILES_PHT}: %.pht: %.ht ${INPUT_EXTRA_INPUT} ${SMAN} \
>> $(fricas_target_bindir)/hypertex$(EXEEXT) stamp-initial-ht-db
>> echo $<
>> unset DAASE; HTPATH=.; export HTPATH; \
>> FRICAS_INITFILE='' ${XVFB} ${SMAN} -noihere -paste $<
>>
>> mobius.VIEW: ${inputsrcdir}/mobius.input
>> (unset DAASE; FRICAS=${FRICAS}; export FRICAS; \
>> FRICAS_INITFILE='' ${XVFB} \
>> ${FRICAS}/bin/fricas -noht -noclef -eval ')read $<')
>> ===============
>>
>> in src/doc/Makefile.in needs src/doc/ht/util.ht?
>
> Well, first rule needs 'util.ht'. I did not check which one.
> 'mobius.VIEW' can be generated without using 'util.ht'.

That is true according to the rules that are currently in
src/doc/Makefile.in. However, I've just built fricas, then removed
util.ht from the build directory and also from the source directory.
Additionally I remove "util" from the line in Makefile (in the build
directory) that corresponds to

https://github.com/fricas/fricas/blob/master/src/doc/Makefile.in#L237

Then said "cd src/doc; make book.pdf" and got book.pdf with all
generated pictures.

So we are both right.

I could change Makefile.in to make it really independent of util.ht.
In fact, now that we discuss about it. Since it is actually a style file
for hyperdoc, it *is* in that sense special. I only left it under
src/doc to keep all .ht files together. When I developed the build
machinery for book.pdf, I had no idea about what was used where.

Now it seems that the better place for it would be together with
hyperdoc. "util.ht" is needed by hyperdoc to interpret the .ht file.
If src/doc/ht somewhere mentions util.ht so that it is clear how to
write a new .ht file, then moving util.ht to src/hyperdoc or src/interp
would make sense.

Leaving it in src/doc is only a reminder that ht.awk should generate
only commands that are in util.ht.

I do not feel big pressure to move util.ht now, but would like to keep
the comment as I proposed, just as a reminder to myself.

Ralf

[Waldek Hebisch]

On Wed, Dec 27, 2023 at 10:21:08PM +0100, Ralf Hemmecke wrote:
> > > Are you saying that
> > >
> > > ================
> > > ${GEN_VIEWPORTFILES_PHT}: %.pht: %.ht ${INPUT_EXTRA_INPUT} ${SMAN} \
> > > $(fricas_target_bindir)/hypertex$(EXEEXT) stamp-initial-ht-db
> > > echo $<
> > > unset DAASE; HTPATH=.; export HTPATH; \
> > > FRICAS_INITFILE='' ${XVFB} ${SMAN} -noihere -paste $<
> > >
> > > mobius.VIEW: ${inputsrcdir}/mobius.input
> > > (unset DAASE; FRICAS=${FRICAS}; export FRICAS; \
> > > FRICAS_INITFILE='' ${XVFB} \
> > > ${FRICAS}/bin/fricas -noht -noclef -eval ')read $<')
> > > ===============
> > >
> > > in src/doc/Makefile.in needs src/doc/ht/util.ht?
> >
> > Well, first rule needs 'util.ht'. I did not check which one.
> > 'mobius.VIEW' can be generated without using 'util.ht'.
>
> That is true according to the rules that are currently in
> src/doc/Makefile.in. However, I've just built fricas,

If you have xvfb, then build already generated images (more precisely
.xpm files). .xpm files are generated by cooperation of FriCASsys
and graphis and in this sense do not need 'util.ht'. But commands
to FriCASsys are sent from HyperDoc and HyperDoc needs macros from
'util.ht' to do this. In other words, generating .xpm files is
a byproduct of normal generation of HyperDoc pages which needs
'util.ht'.

> then removed util.ht
> from the build directory and also from the source directory. Additionally I
> remove "util" from the line in Makefile (in the build directory) that
> corresponds to
>
> https://github.com/fricas/fricas/blob/master/src/doc/Makefile.in#L237
>
> Then said "cd src/doc; make book.pdf" and got book.pdf with all generated
> pictures.

In this stage you merely convert .xpm files to .ps.

> I could change Makefile.in to make it really independent of util.ht.
> In fact, now that we discuss about it. Since it is actually a style file for
> hyperdoc, it *is* in that sense special. I only left it under src/doc to
> keep all .ht files together. When I developed the build machinery for
> book.pdf, I had no idea about what was used where.

Generating HyperDoc pages is documented, so all needed to do
this must be available in shipped tarball. So we need to copy
util.ht from source location to destination. Assuming that we
always use 'util.ht' from target tree its exact location in
source tree is not that important. But it makes sense to keep
it together with other .ht files. If somewhere we use 'util.ht'
from source tree, then this is a bit unclean, but normally they
are the same so not too bad.

And 'util.ht' is not so special: you can define macros in normal
files but normally we want to share macros for all files, so
we put them in a common place. And changing HyperDoc pages
may need new macro, so it is really better to keep them together.

--
Waldek Hebisch

[Waldek Hebisch]

On Wed, Dec 27, 2023 at 06:23:05PM +0800, Qian Yun wrote:

Using '$spadroot' in sources we need to call '(initroot)' first,
so we should do the above.

But thinking about HyperDoc, I see another problem: FriCAS is
supposed to use current 'util.ht' not a cached copy from the
past. Long ago Spad compiler was autoloaded and 'util.ht' was
read as a side effect. Now all files are included in the
image and side effect take place during image build. We should
defer call to 'buildHtMacroTable()', either do this at FriCAS
startup or on first use of ')compile' (I think it is used only
for compilation).
--
Waldek Hebisch

[Qian Yun]

On 12/28/23 07:31, Waldek Hebisch wrote:
>
> But thinking about HyperDoc, I see another problem: FriCAS is
> supposed to use current 'util.ht' not a cached copy from the
> past. Long ago Spad compiler was autoloaded and 'util.ht' was
> read as a side effect. Now all files are included in the
> image and side effect take place during image build. We should
> defer call to 'buildHtMacroTable()', either do this at FriCAS
> startup or on first use of ')compile' (I think it is used only
> for compilation).

(7) -> )lisp (time (|buildHtMacroTable|))

Evaluation took:
0.000 seconds of real time
0.000511 seconds of total run time (0.000409 user, 0.000102 system)
100.00% CPU
1 form interpreted
5,297,256 processor cycles
159,104 bytes consed

Value = #<HASH-TABLE :TEST EQUAL :COUNT 271 {100240C353}>

So the performance is totally acceptable for us to run this function
at each startup.

"interpsys_restart" should be a good place to put it?

- Qian

[Waldek Hebisch]

Yes.

--
Waldek Hebisch

[Qian Yun]

Let's review this version.

Tested with image generation, no problems.

"buildHtMacroTable" is before "spadStartUpMsgs", so even if the
warning message is printed, it is printed before banner.

- Qian

diff --git a/src/doc/Makefile.in b/src/doc/Makefile.in

index e968affd..19333585 100644
--- a/src/doc/Makefile.in
+++ b/src/doc/Makefile.in
@@ -255,7 +255,6 @@ GEN_HTFILES = xmpexp coverex gloss

# The reason for this delayed execution is that generation of the .xpm
# files in *.VIEW/ without the xvfb-run program will pop up image
# windows in X and will make working during compilation impossible.
-# The intro.htex and util.ht files are not relevant for image generation.

VIEWPORTFILES = coverex \
$(shell grep -lRe 'graphpaste{\|spadgraph{' ${srcdir}/ht/*
${srcdir}/htex/* \
| sed 's,.*/,,;s,\..*,,' \

diff --git a/src/interp/htcheck.boot b/src/interp/htcheck.boot
index e3f78805..47fa0253 100644
--- a/src/interp/htcheck.boot
+++ b/src/interp/htcheck.boot
@@ -125,5 +125,3 @@ spadSysBranch(tree,arg) == --tree is (msg kind
TREEorSomethingElse ...)
kind = 'INTEGER => INTEGERP arg
kind = 'FUNCTION => atom arg
systemError '"unknown tree branch"
-
-buildHtMacroTable()
diff --git a/src/interp/i-toplev.boot b/src/interp/i-toplev.boot
index 20b4773c..0b2250b0 100644
--- a/src/interp/i-toplev.boot
+++ b/src/interp/i-toplev.boot
@@ -94,6 +94,7 @@ interpsys_restart() ==
loadExposureGroupData()
initHist()
initializeInterpreterFrameRing()
+ buildHtMacroTable()

if $displayStartMsgs then spadStartUpMsgs()
$currentLine := nil

[Waldek Hebisch]

On Thu, Dec 28, 2023 at 08:47:30AM +0800, Qian Yun wrote:
> Let's review this version.
>
> Tested with image generation, no problems.
>
> "buildHtMacroTable" is before "spadStartUpMsgs", so even if the
> warning message is printed, it is printed before banner.

Looks OK.

--
Waldek Hebisch