Building API web pages

[Waldek Hebisch]

On my new machine building API web pages takes twice as much
real time as normal FriCAS build. This is due to sphinx
taking a lot of time. I have tried to modify Ralf's code
to directly generate HTML, skipping the .rst stage. You
can see proof of concept code at:

http://wiki.fricas.org/public/api2.spad

ATM this does not handle docstrings and conditions on signatures.
Navigation links are very limited. But it indicates that
skipping .rst stage we should be able to build API web pages
probably about 100 times faster.

If you want to try this, first generate 'abbrev-lines.list'
(you can take it from documentation build), then compile
'api.spad' from 'src/doc', then compile 'api2.spad' and
at FriCAS prompt issue

documentAll("abbrev-lines.list")

For some reason second second run is much faster than the
first one. Since this generates thousends of pages in
reasonable time, we should be able to generate pages on
the fly, making this dynamic (a not only having static
pages).

BTW: Sphinx-generated HTML contains a lot of markup. I wonder
how much of it is useful (for us) and how much is crap added
because in some rare cases (but usually not) it is needed.

--
Waldek Hebisch

[Ralf Hemmecke]

On 12/2/24 00:38, Waldek Hebisch wrote:
> http://wiki.fricas.org/public/api2.spad

I'd like to wait with my feedback until I see others (especially those
that USE https://fricas.github.io/api) say an opinion about the produced
output from api2.spad.

Ralf

[Grégory Vanuxem]

Hello,

Le lun. 2 déc. 2024 à 18:07, 'Ralf Hemmecke' via FriCAS - computer
algebra system <fricas...@googlegroups.com> a écrit :

I hope it's not about me.

To speak apart I find this very interesting, I tried it, yes, it does
not support docstrings for example, my principal concern in this area,
but I looked at the api.spad code and it is too heavy for me. I looked
at it to see if it was possible to add something like a plugin/addon
to support MarkDown. I am not for or against restructuredText or
Markdown, just I little like the LaTeX syntax in MarkDown. Of course
it is not LaTeX, just a very _small_ subset, but from a pragmatic
point of view I see this simpler to use it to improve documentation in
panAxiom. Furthermore via Jekyll it is supported by GitHub so it is a
plus.

To say the truth, what I think deep inside me, it's more a proof of
concept I think. I agree Sphinx doc adds tons of build time, I know I
am used before committing something in my repository but on the other
hand I like the idea of having different formats but Sphinx generated
documentation is something "relatively" beautiful.

- Greg

PS: I have seen the discussion on the utility or not to use
restructuredText for the README file but at that time I was totally
novice in this area. I am still a novice.


> Ralf
>
> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/fricas-devel/95a7f715-479b-4622-a511-bf3a8c4e96e7%40hemmecke.org.

[Waldek Hebisch]

ATM api2.spad output can not serve as a replacement for output of
api.spad+sphinx, it still needs implementation of few essential features.

Concerning api.spad+sphinx markup, it seem to me that 'edit on github'
link is somewhat out of place for 'localdoc' (I personally do not use
it at all, but maybe other people use it). Actually, I do not see
it in Firefox (it shows up in Lynx), so it is not clear to me how
it is supposed to work (I do not think that Lynx is main browser
for viewing API pages).

I also noticed that when translating something like '\spad{reduce(f, x, r)}'
current output seem to be

<code class="docutils literal notranslate"><span class="pre">reduce(f,</span>
<span class="pre">x,</span> <span class="pre">r)</span></code>

that is we get 3 separate 'span'-s. I think it would be better to
have

<code class="docutils literal notranslate">
<span class="pre">reduce(f, x, r)</span></code>

or possibly even

<code>reduce(f, x, r)</code>

Another things are self-links, do they serve any useful purpose?
Currently each generated page contains 3 sets of navigations links,
one in header, one at start of the page (IIUC this one is to generate
access keys) and one at the end. Second and third set contain self-link.

Another thing is 'permalinks'. IIUC a 'permalink' is supposed to
be a stable link, reliably giving "the same page". But as api
site is unversioned any link to it is really not a 'permalink'.
So, do they make any sense on generated pages?

Another things is estetics versus usability. Currently in
Firefox main part of a page is divided into 3 strips. Left
strip shows essentially the same thing on all pages, right is
empty (IIUC main strip has limited width and the third strip
only shows up when window is wide enough) and actual information
is in the middle strip. When there is not much information and
one has big screen, then this layout looks nice. But it wastes
about 30% of screen area, which makes it much less usable on small
screens.

--
Waldek Hebisch

[Ralf Hemmecke]

On 12/4/24 02:20, Waldek Hebisch wrote:
> On Mon, Dec 02, 2024 at 06:06:54PM +0100, 'Ralf Hemmecke' via FriCAS
> - computer algebra system wrote:
>> On 12/2/24 00:38, Waldek Hebisch wrote:
>>> http://wiki.fricas.org/public/api2.spad
>>
>> I'd like to wait with my feedback until I see others (especially
>> those that USE https://fricas.github.io/api) say an opinion about
>> the produced output from api2.spad.
>
> ATM api2.spad output can not serve as a replacement for output of
> api.spad+sphinx, it still needs implementation of few essential
> features.

Maybe our goals differ here. I want nice, attractive, heavily
hyperlinked and searchable documentation. Build-time is not my primary
focus. I agree that it takes long and probably sphinx it doing needless
scanning of the .rst files for whatever reason. I bet that there might
be an option to speed that up for our particular case. I was new to
sphinx when I started. I still believe it is better than translating
directly to HTML. I don't want to prevent you from continuing with your
direct .spad -> .html translation for nightly builds, but I would like
to keep the sphinx-generated output.

> Concerning api.spad+sphinx markup, it seem to me that 'edit on

> github' link is somewhat out of place for 'localdoc'. (I personally

> do not use> it at all, but maybe other people use it).

Well, having such a link might lower the barrier for people reading the
documentation and spotting a typo. Just click, edit, and create a pull-
request. And that can also work with the locally generated code. Of
course you need an internet connection when you click on "edit on
github", but I hope that you see the intent.

> Actually, I do not see it in Firefox (it shows up in Lynx), so it is
> not clear to me how it is supposed to work (I do not think that Lynx
> is main browser for viewing API pages).

Huh? Note that there are two different types of source for fricas.github.io.
(1) .rst files
(2) .spad files

For (1) there appears a "edit on github" link in the left column and
points to the .rst source on github.
For (2) I couldn't do that, since linking to the generated .rst file
makes no sense. I therefore generate a link that points to the
respective .spad file. And that link also appears at the top of each API
page, looking like "any.spad line 197 [edit on github]". And this also
is shown in Firefox. I don't understand why you would not see it.

> I also noticed that when translating something like '\spad{reduce(f,
> x, r)}' current output seem to be
>
> <code class="docutils literal notranslate"><span
> class="pre">reduce(f,</span> <span class="pre">x,</span> <span
> class="pre">r)</span></code>
>
> that is we get 3 separate 'span'-s. I think it would be better to
> have

> <code class="docutils literal notranslate"> <span
> class="pre">reduce(f, x, r)</span></code>
>
> or possibly even
>
> <code>reduce(f, x, r)</code>

OK, that is real critisism and I agree. In fact, the rst2html translator
coming with sphinx does not introduce that.

First step to make the output nicer is to translate

\spad{foo}

into :code:`foo` instead of the current ``foo``. I have already a patch
that does that.

That splitting that you mentioned obviously happens because there are
spaces inside the :code:`foo bar` source. Some other transformation that
happens in sphinx is doing this splitting. I guess it is connected to
highlighting of inline code. We currently need no highlighting, so I
will be looking for a way to switch that off. Help is welcome.

> Another things are self-links, do they serve any useful purpose?

What is your definition of "self-links"?
Do you mean the bread-crumbs line that shows the user where he is in the
hierarchy?

> Another thing is 'permalinks'. IIUC a 'permalink' is supposed to be
> a stable link, reliably giving "the same page". But as api site is
> unversioned any link to it is really not a 'permalink'. So, do they
> make any sense on generated pages?

Do you mean a link like this?

http://fricas.github.io/api/BasicType.html#basictype

That should be permanent as long as there is BasicType in FriCAS.

> Another things is estetics versus usability. Currently in Firefox
> main part of a page is divided into 3 strips.

> When there is not much information and one has big screen,> then this
layout looks nice. But it wastes about 30% of screen
> area, which makes it much less usable on small screens.

OK, accepted. But there are other themes. For example, you can cange to

html_theme = 'alabaster'

in the file src/doc/tmp/sphinx/source/conf.py inside the build directory.

cd $builddir/src/doc/tmp/sphinx
rm -r build
make html
firefox build/html/index.html

That gives you output with responsive design and even the breadcrumbs
lines are gone. Maybe alabaster is not the best theme. I like the one
used here better...

https://doc.sagemath.org/html/en/reference/index.html

Does anyone know what theme they use?

Ralf

[Waldek Hebisch]

On Thu, Dec 05, 2024 at 09:31:00PM +0100, 'Ralf Hemmecke' via FriCAS - computer algebra system wrote:
> On 12/4/24 02:20, Waldek Hebisch wrote:
> > On Mon, Dec 02, 2024 at 06:06:54PM +0100, 'Ralf Hemmecke' via FriCAS
> > - computer algebra system wrote:
> > > On 12/2/24 00:38, Waldek Hebisch wrote:
> > > > http://wiki.fricas.org/public/api2.spad
> > >
> > > I'd like to wait with my feedback until I see others (especially
> > > those that USE https://fricas.github.io/api) say an opinion about
> > > the produced output from api2.spad.
> >
> > ATM api2.spad output can not serve as a replacement for output of
> > api.spad+sphinx, it still needs implementation of few essential
> > features.
>
> Maybe our goals differ here. I want nice, attractive, heavily
> hyperlinked and searchable documentation. Build-time is not my primary
> focus.

Well, I prefer dynamically generated pages. I have tried a bit
search on API site and comparison with HyperDoc is that
HyperDoc clearly wins: HyperDoc offers several views with
filtering options, so it is not hard to get to desired info.
API search lumped thing together in rather inconvenient way.
API search probably could be improved, but I do not see possiblity
of offering customised views (like HyperDoc) in mostly static
site without creating inpracticaly large number of pages.
Theoretically you can re-implement HyperDoc functinality in
Javascript and do computations in the browser, but this seem
to be incompatible with Sphinx. And IMO undesirable, as this
would duplicate code that we already have and need.

So, IMO, good search needs dynamic "server-side" generated pages.

Another issue is testing. Ideally building API pages should
be routine part of building and testing. But this is problematic
both due to long time and because Sphinx has many dependencies.

Another thing is that I would avoid having many versions of code
that are doing similar things, but where no version is a complete
solution. Rather, I would like to have a single thing which
can produce classic HyperDoc pages, dynamic Html or (I you wich)
restuctured text.

> I agree that it takes long and probably sphinx it doing needless
> scanning of the .rst files for whatever reason. I bet that there might
> be an option to speed that up for our particular case. I was new to
> sphinx when I started. I still believe it is better than translating
> directly to HTML. I don't want to prevent you from continuing with your
> direct .spad -> .html translation for nightly builds, but I would like
> to keep the sphinx-generated output.
>
> > Concerning api.spad+sphinx markup, it seem to me that 'edit on
> > github' link is somewhat out of place for 'localdoc'. (I personally
> > do not use> it at all, but maybe other people use it).
>
> Well, having such a link might lower the barrier for people reading the
> documentation and spotting a typo. Just click, edit, and create a pull-
> request. And that can also work with the locally generated code. Of
> course you need an internet connection when you click on "edit on
> github", but I hope that you see the intent.

I suspect that it is not useful for 99.9% of people. And for
remaing ones it saves maybe 0.1% of work needed to create
sensible pull request. Simply, it is not clear if it is worth
file and screen space.

> > Actually, I do not see it in Firefox (it shows up in Lynx), so it is
> > not clear to me how it is supposed to work (I do not think that Lynx
> > is main browser for viewing API pages).
>
> Huh?

Sorry, it is there. Simply too many links have distracted me and
I somewhat "turned off".

> > I also noticed that when translating something like '\spad{reduce(f,
> > x, r)}' current output seem to be
> >
> > <code class="docutils literal notranslate"><span
> > class="pre">reduce(f,</span> <span class="pre">x,</span> <span
> > class="pre">r)</span></code>
> >
> > that is we get 3 separate 'span'-s. I think it would be better to have
>
> > <code class="docutils literal notranslate"> <span
> > class="pre">reduce(f, x, r)</span></code>
> >
> > or possibly even
> >
> > <code>reduce(f, x, r)</code>
>
> OK, that is real critisism and I agree. In fact, the rst2html translator
> coming with sphinx does not introduce that.
>
> First step to make the output nicer is to translate
>
> \spad{foo}
>
> into :code:`foo` instead of the current ``foo``. I have already a patch that
> does that.
>
> That splitting that you mentioned obviously happens because there are spaces
> inside the :code:`foo bar` source. Some other transformation that happens in
> sphinx is doing this splitting.

Yes, code has and should have spaces. With proper HTML generator
that should be easy.

> I guess it is connected to highlighting of
> inline code. We currently need no highlighting, so I will be looking for a
> way to switch that off. Help is welcome.
>
> > Another things are self-links, do they serve any useful purpose?
>
> What is your definition of "self-links"?
> Do you mean the bread-crumbs line that shows the user where he is in the
> hierarchy?

Those too.

> > Another thing is 'permalinks'. IIUC a 'permalink' is supposed to be
> > a stable link, reliably giving "the same page". But as api site is
> > unversioned any link to it is really not a 'permalink'. So, do they
> > make any sense on generated pages?
>
> Do you mean a link like this?
>
> http://fricas.github.io/api/BasicType.html#basictype
>
> That should be permanent as long as there is BasicType in FriCAS.

Well, this is really a self-links and AFAICS its permanence is no
better or worse than any other link inside the site. OK, BasicType
is likely to be there for long time, but several constructors are
gone and more will go away.

Such links make sense in a blog where top page changes frequently
and permalink allows to find the same page. Or in case of
versioned documentation, where documentation for version 24.12
contains links which point to 24.12, even when newer version
is current.

But in our case I see no usefule purpose: link that led to
page (or link shown by browser) will work as well. If you
think about spiders, I looked at logs of wiki site and
spiders happily index dynamic pages, so it seems to have
no real impact on them.

> > Another things is estetics versus usability. Currently in Firefox
> > main part of a page is divided into 3 strips.
>
> > When there is not much information and one has big screen,> then this
> layout looks nice. But it wastes about 30% of screen
> > area, which makes it much less usable on small screens.
>
> OK, accepted. But there are other themes. For example, you can cange to
>
> html_theme = 'alabaster'
>
> in the file src/doc/tmp/sphinx/source/conf.py inside the build directory.
>
> cd $builddir/src/doc/tmp/sphinx
> rm -r build
> make html
> firefox build/html/index.html
>
> That gives you output with responsive design and even the breadcrumbs lines
> are gone. Maybe alabaster is not the best theme. I like the one used here
> better...
>
> https://doc.sagemath.org/html/en/reference/index.html
>
> Does anyone know what theme they use?

Trying various themes could be quite time consuming. It looks
easier to collect useful pieces of HTML/Javascript do get
something that we like (one reason for my question was that this
would be easier if you or somebody else could say "I want this
feature and this pieces of HTML gives it").

--
Waldek Hebisch

[Martin Baker]

I haven't been following this thread too carefully so ignore this if
I've gone off at a tangent.
I think you are discussing requirements for HTML documentation so here
are my requirements:

I would like it to work reliably on any browser and any device (PC,
mobile, etc.) even when browsers change. For me static web pages are a
safer bet in this regard and would require a lot less maintenance when
things change than say javascript.

I would like to look at FriCAS as a toolkit and I would like the
documentation to give me an up-to-date overview of all the mathematical
structures supported and then to be able to drill down to the details.

I think its important that the documentation looks modern and supports
the conventions people expect and not a clunky old interface like
HyperDoc. I guess that's not important to the people on this list but I
think it will be frightening away new users.

Of all the fragmented documentation for FriCAS I use
https://fricas.github.io/ as it best meets my requirements. I get the
impression that Waldek is very negative about this documentation, I
think that's a pity as I would like to see it extended and the other
documentation merged into it.

Just my 2 cents.

Martin Baker

[Waldek Hebisch]

On Sun, Dec 08, 2024 at 11:06:48AM +0000, Martin Baker wrote:
> I haven't been following this thread too carefully so ignore this if
> I've gone off at a tangent.
> I think you are discussing requirements for HTML documentation so here
> are my requirements:
>
> I would like it to work reliably on any browser and any device (PC,
> mobile, etc.) even when browsers change. For me static web pages are a
> safer bet in this regard and would require a lot less maintenance when
> things change than say javascript.

I wrote about dynamic "server-side" generated pages. I agree that
javascript may be problematic, that is one reason to do dynamic
part on server side: from client point of view this is the same
as static pages. Simply, instead of pr-generating inpractically
large number of pages sever may generate them on the fly.

> I would like to look at FriCAS as a toolkit and I would like the
> documentation to give me an up-to-date overview of all the mathematical
> structures supported and then to be able to drill down to the details.
>
> I think its important that the documentation looks modern and supports
> the conventions people expect and not a clunky old interface like
> HyperDoc.

Well, I prefer to distinguish presentation form logic. IMO in logic
aspect HyperDoc is reasonably modern, main "logic" drawback of current
HyperDoc is that it gives unrestricted access to FriCAS, so it
can not be directly exposed to internet (at least not without extra
protection). I think that for API server use this can be fixed with
resonable effort, so the FriCAS only presents API and nothing more.

Concerning presentation, I want to offer HTML access, so that
the same content that is currently available via HyperDoc can be
accessed from web browser. If you look deeper, beyond "clunky"
and "old" you should see that in current documentation there is page
structure, links, etc, we just need to map the constructs to HTML.
There are features that are harder to map, like "live" execution
of examples.

> I guess that's not important to the people on this list but I
> think it will be frightening away new users.
>
> Of all the fragmented documentation for FriCAS I use
> https://fricas.github.io/ as it best meets my requirements. I get the
> impression that Waldek is very negative about this documentation, I
> think that's a pity as I would like to see it extended and the other
> documentation merged into it.

Well, concerning API pages: negative part is that I do not see how
we can get good search using current technology (that is Sphinx).
Also, I find build time problematic. I would like to resolve those
problems, and I think that direct generation of HTML can resolve
both problems. So, my intent is to extend and improve API
pages.

I also wrote about HTML constructs. My low-level view of API
pages is that there is bunch of templates (if order of 10) with
slots that are filled with info. I consider generating info which
goes into slots as main task (rest is basically a lot of string
concatenations). Current 'api2.spad' uses very minimal templates,
which are adequate to debug correctness of information, but are
not intended as end product. And I wrote, work on generating
info is not finished, there are docstrings to handle, conditions
and sorting. However, there is also question of HTML templates.
In principle we could just copy templates used by Sphinx. But
this in unlikely to be optimal choice. That is reason for my
HTML questions. End-user opinions about look of pages are
important, but since 'api2.spad' is at early stage and look
is quite different from what it can be my question was mostly
to people developing web pages. That is Ralf, you, Bill Page,
Arthur Ralfs (I do not know if they still read FriCAS list)
and possibly others.

--
Waldek Hebisch

[Martin Baker]

On 13/12/2024 16:37, Waldek Hebisch wrote:
> I wrote about dynamic "server-side" generated pages. I agree that
> javascript may be problematic, that is one reason to do dynamic
> part on server side: from client point of view this is the same
> as static pages. Simply, instead of pr-generating inpractically
> large number of pages sever may generate them on the fly.

That looks ambitious to me but I don't know much about server software.
Would it run on Github or would it require use of a university server
somewhere?
Something makes me uneasy about this, I'm just thinking back to the
issues I saw on this thread about the axiom-wiki. I thought that server
software was high maintenance due to all the things that can change such
as: 'new version of Debian', 'hard disk dying', and so on.
I am just worried about the FriCAS documentation disappearing off the
web every time there is a software glitch.
Would it be possible to retain https://fricas.github.io/ for what it
does now and just link to the dynamic pages for things like: searching
function names or live execution of examples.

In practice I wonder how useful live execution of examples is? Its hard
enough/impossible to get people to write even minimal documentation and
this live execution would be a lot more work on top of that. So
realistically, is it likely to increase beyond the small number of
examples already in HyperDoc?

> Well, I prefer to distinguish presentation form logic. IMO in logic
> aspect HyperDoc is reasonably modern, main "logic" drawback of current
> HyperDoc is that it gives unrestricted access to FriCAS, so it
> can not be directly exposed to internet (at least not without extra
> protection). I think that for API server use this can be fixed with
> resonable effort, so the FriCAS only presents API and nothing more.

I don't understand this. What would it give access to? the source code?

> Concerning presentation, I want to offer HTML access, so that
> the same content that is currently available via HyperDoc can be
> accessed from web browser. If you look deeper, beyond "clunky"
> and "old" you should see that in current documentation there is page
> structure, links, etc, we just need to map the constructs to HTML.
> There are features that are harder to map, like "live" execution
> of examples.

Yes, I really want to stress how important presentation is for new
users. Not just having HTML but having it nicely formatted for different
platforms like mobile.

> Well, concerning API pages: negative part is that I do not see how
> we can get good search using current technology (that is Sphinx).
> Also, I find build time problematic. I would like to resolve those
> problems, and I think that direct generation of HTML can resolve
> both problems. So, my intent is to extend and improve API
> pages.
>
> I also wrote about HTML constructs. My low-level view of API
> pages is that there is bunch of templates (if order of 10) with
> slots that are filled with info. I consider generating info which
> goes into slots as main task (rest is basically a lot of string
> concatenations). Current 'api2.spad' uses very minimal templates,
> which are adequate to debug correctness of information, but are
> not intended as end product. And I wrote, work on generating
> info is not finished, there are docstrings to handle, conditions
> and sorting. However, there is also question of HTML templates.
> In principle we could just copy templates used by Sphinx. But
> this in unlikely to be optimal choice. That is reason for my
> HTML questions. End-user opinions about look of pages are
> important, but since 'api2.spad' is at early stage and look
> is quite different from what it can be my question was mostly
> to people developing web pages. That is Ralf, you, Bill Page,
> Arthur Ralfs (I do not know if they still read FriCAS list)
> and possibly others.

So would these templates be automatically populated from the source code
and the '++' comments? It seems to me that there are already too many
sources of out-of-step documentation so I guess you want all the
official documentation to be derived from some common root?

Martin

[Waldek Hebisch]

On Fri, Dec 13, 2024 at 06:47:29PM +0000, Martin Baker wrote:
> On 13/12/2024 16:37, Waldek Hebisch wrote:
> > I wrote about dynamic "server-side" generated pages. I agree that
> > javascript may be problematic, that is one reason to do dynamic
> > part on server side: from client point of view this is the same
> > as static pages. Simply, instead of pr-generating inpractically
> > large number of pages sever may generate them on the fly.
>
> That looks ambitious to me but I don't know much about server software.
> Would it run on Github or would it require use of a university server
> somewhere?

For server-side generated pages you need ability to run sofware of
your choice on the server machine. That excludes Github or more
generally various "web hosting" offers, requireing running it locally
on user machine or a "dedicated server", like 'fricas.org'. This does
not exclude possibility of hosting pre-generated pages in Github,
simply pages generated on the fly would be absent from Github site.

> Something makes me uneasy about this, I'm just thinking back to the
> issues I saw on this thread about the axiom-wiki. I thought that server
> software was high maintenance due to all the things that can change such
> as: 'new version of Debian', 'hard disk dying', and so on.
> I am just worried about the FriCAS documentation disappearing off the
> web every time there is a software glitch.

'axiom-wiki' has three releated problem:
- it depends on LaTeX wiki software, which is unmaintained for
many years,
- via LaTeX wiki it depends on Zope which stores data as a binary
blob which only Zope can read,
- Zope rather frequently changes in incompatible ways.

Due to above there would be substantial work to move to newer
version on Zope and we are currently stuck with very old version.

For some time it looked hard to upgrade system on the machine,
as Zope depends on specific old version of Python, but this part
is resolved now, I identified want we need and can install old
Zope and old Python on a new system.

Of course 'hard disk dying' may happen, but actually one disc
dying is not a big trouble, one simply replaces the drive. Trouble
was two discs failing in short interval (like two weeks) during
holiday time, so there was nobody in place to quickly replace failed
disc. And at core of trouble was faulty series of disc drives,
similar failures affected other machines later (but discs were
replaced quickly enough to avoid data loss).

> Would it be possible to retain https://fricas.github.io/ for what it
> does now and just link to the dynamic pages for things like: searching
> function names or live execution of examples.

Yes.

> In practice I wonder how useful live execution of examples is? Its hard
> enough/impossible to get people to write even minimal documentation and
> this live execution would be a lot more work on top of that. So
> realistically, is it likely to increase beyond the small number of
> examples already in HyperDoc?

Not so small if you count FriCAS book. Concerning usefulness, I
think that this is useful, but I want to get other things working
with HTML first, especially search. I mentioned live execution
as a thing which more complicated that other things, at least
when you want this working over internet.

> > Well, I prefer to distinguish presentation form logic. IMO in logic
> > aspect HyperDoc is reasonably modern, main "logic" drawback of current
> > HyperDoc is that it gives unrestricted access to FriCAS, so it
> > can not be directly exposed to internet (at least not without extra
> > protection). I think that for API server use this can be fixed with
> > resonable effort, so the FriCAS only presents API and nothing more.
>
> I don't understand this. What would it give access to? the source code?

Source code access is just links, no trouble. Claryfing a bit,
currenly HyperDoc implement large part of its functionality by
sending commands to FRICASsys. Those commands can do essentially
anything, basicaly can upload new code to FRICASsys and using
system calls do anyting that Unix process with permissions of
FRICASsys can do. That is OK if FRICASsys and cource of commands
runs on the same machine (that is both run on user machine) with
no access fromoutside. But this is security trouble, when
we want access from internet. For this we need to restrict
available commands to some allowed subset. API pages like
current pages in Github pose no trouble, they are easy to
recognize and validate. Search as done currently in HyperDoc
is more tricky, as HyperDoc allows arbitrary expressions
in type parameters. But AFAICS most general forms are rarely
used and if we exclude most general cases the rest should be
easy to filter.

HyperDoc currently handles FriCAS book in C code (FRICASsys
is only involved in live examples). 'api.spad' contains
convertor from LaTeX susbset used in docstings to Restructured
Text. IMO the approach will not scale up to more complex
tasks, so I want a different convertor. I would like new
convertor to handle FriCAS book.

Yes, exactly.

--
Waldek Hebisch

[Ralf Hemmecke]

On 12/14/24 00:00, Waldek Hebisch wrote:
>>> as static pages. Simply, instead of pr-generating inpractically
>>> large number of pages sever may generate them on the fly.

Maybe, there are different people that have different goals...

Perhaps it would be useful to clarify what each of us wants and perhaps
it makes more sense not to work against each other. To say the truth, at
the moment I feel as if you do not value the work that I put into
fricas.github.io and rather aim at creating your own way to generate
documentation and get rid of what I did. I hope you understand that this
wouldn't make me happy.

Yes. I do not like HyperDoc and this is just because it feels like old
technology and *I* do not want to spend time on it. But FriCAS is an
open-source project, so everyone can do whatever he/she likes.

Not liking HyperDoc does not mean that I am against a system that
generates API pages from within a FriCAS session locally.

When I started with the sphinx pages, I had the explicit goal to bring
the API as static pages to the web. Goal 1: more visibility of FriCAS.
Goal 2: searchable by this sphinx generated search field and also by any
search engine. For example enter "Module site:fricas.github.io" into google.

Yes, sure HyperDoc is better when it comes to specifying domain/
category/package parameters and retrieve information for such particular
case. Still, HyperDoc looks so old-fashioned that I basically never use
it. If I could do the same stuff locally with my webbrowser, it would
improve the situation.

I basically see need for (1) static web pages (versioned for each
release) and (2) a local help/documentation system that helps from
within a FriCAS session (even without Internet access).

I would like to work on (1). Generation time of such web pages is not a
high priority. I have anyway a local service running that checks whether
the webpages still compile fine each time someone commits
something to the official git-repo.
You mentioned a few issues like to many <pre> stuff etc. or sidebars
that use too much space. Well, yes, that can and should be improved, but
that doesn't mean that we/you should *replace* the sphinx generated
pages by server-side generated HTML pages. If you want to create such a
system, do so, but do not ask for removing static pages (I somehow have
the impression that such would be your long term goal -- if it is not
then I apologise).

As for execution over the web. Yes that is dangerous, but if you find a
way to provide it, then do so. My approach rather is to provide a git-
repository of notebooks (for example
https://fricas.github.io/fricas-notebooks/FriCAS-LinearAlgebra.html)
that people can see and download and then modify.
Actually, jFriCAS can probably be made to provide an online FriCAS
service. I haven't yet experimented with it, because also that might
require to start a virtual machine on a server so that no harm can be
done to the server itself. At the moment static notebooks are a good
"first step".

> 'axiom-wiki' has three releated problem: - it depends on LaTeX wiki
> software, which is unmaintained for many years, - via LaTeX wiki it
> depends on Zope which stores data as a binary blob which only Zope
> can read, - Zope rather frequently changes in incompatible ways.

I think FriCAS has too many different forms of documentation. I would
rather be in favour of migrating to just one or two forms and thus
reduce the maintenance work. But as long as there is someone (other than
me) willing to invest time, even that is not a big problem for me.

>> Would it be possible to retain https://fricas.github.io/ for what
>> it does now and just link to the dynamic pages for things like:
>> searching function names or live execution of examples.
>
> Yes.

Yes, that sounds like a good approach.

>> In practice I wonder how useful live execution of examples is?
>> Its hard enough/impossible to get people to write even minimal
>> documentation and this live execution would be a lot more work on
>> top of that. So realistically, is it likely to increase beyond
>> the small number of examples already in HyperDoc?

> Not so small if you count FriCAS book.

Eventually, we should also have the FriCAS book in HTML format on the
web and maybe even with links to notebooks that one can download and
execute.

> Concerning usefulness, I think that this is useful, but I want to
> get other things working with HTML first, especially search.

Can you be a bit more precise? Do you mean for example "search for
exported functions in Foo(R) where a user can set R==Integer" or "search
for the domain/package that implements function foo from domain Foo(R)
where R is Fraction(Integer)"? Such things I would really appreciate and
that wouldn't be a big security issue.

> HyperDoc currently handles FriCAS book in C code (FRICASsys is only
> involved in live examples). 'api.spad' contains convertor from
> LaTeX susbset used in docstings to Restructured Text. IMO the
> approach will not scale up to more complex tasks, so I want a
> different convertor.

What are the "more complex tasks"?

> I would like new convertor to handle FriCAS book.

What exactly is your goal?

>>> Well, concerning API pages: negative part is that I do not see
>>> how we can get good search using current technology (that is
>>> Sphinx).

The question is what you understand by "good search". I have not taken
much effort to tailor the stuff that sphinx produces by default.
I find that search field useful enough (not perfect) to get to the
information I want. Otherwise it might also make sense to just use
google (as mentioned above). And yes, search for specified parameters on
parametrized domains would be super-cool and certainly nothing that
sphinx can or should deliver.

>>> Also, I find build time problematic. I would like to resolve
>>> those problems, and I think that direct generation of HTML can
>>> resolve both problems.

I also think that the build time is too long. If someone can help in
finding the reason for this, I'd appreciate. But removing sphinx would
be a step into the wrong direction. No problem if you provide api2.spad
generated html documentation on fricas.org.

>> So I guess you want all the official documentation to be derived

>> from some common root?

> Yes, exactly.

And for that it would be a good idea to agree upon a format that is both
readable in source code (++ docstrings) and has enough information to
generate good looking webpages from it. I'd really prefer reST over
HyperTex.

Ralf

[Waldek Hebisch]

On Sat, Dec 14, 2024 at 02:38:35AM +0100, 'Ralf Hemmecke' via FriCAS - computer algebra system wrote:
> On 12/14/24 00:00, Waldek Hebisch wrote:
> > > > as static pages. Simply, instead of pr-generating inpractically
> > > > large number of pages sever may generate them on the fly.
>
> Maybe, there are different people that have different goals...
>
> Perhaps it would be useful to clarify what each of us wants and perhaps
> it makes more sense not to work against each other. To say the truth, at
> the moment I feel as if you do not value the work that I put into
> fricas.github.io and rather aim at creating your own way to generate
> documentation and get rid of what I did. I hope you understand that this
> wouldn't make me happy.

Well, fricas.github.io is nice and I value the work you put into it.
OTOH Sphinx IMO is a problematic dependency, so I would like to
have equivalent effect without using Sphinx. Also, I want more
functionality, some which looks impossible to get using Sphinx.

> Yes. I do not like HyperDoc and this is just because it feels like old
> technology and *I* do not want to spend time on it. But FriCAS is an
> open-source project, so everyone can do whatever he/she likes.
>
> Not liking HyperDoc does not mean that I am against a system that
> generates API pages from within a FriCAS session locally.
>
> When I started with the sphinx pages, I had the explicit goal to bring
> the API as static pages to the web. Goal 1: more visibility of FriCAS.
> Goal 2: searchable by this sphinx generated search field and also by any
> search engine. For example enter "Module site:fricas.github.io" into google.
>
> Yes, sure HyperDoc is better when it comes to specifying domain/
> category/package parameters and retrieve information for such particular
> case. Still, HyperDoc looks so old-fashioned that I basically never use
> it. If I could do the same stuff locally with my webbrowser, it would
> improve the situation.
>
> I basically see need for (1) static web pages (versioned for each
> release) and (2) a local help/documentation system that helps from
> within a FriCAS session (even without Internet access).
>
> I would like to work on (1). Generation time of such web pages is not a
> high priority. I have anyway a local service running that checks whether
> the webpages still compile fine each time someone commits
> something to the official git-repo.

We can install dynamically generated pages at fricas.org, so this
is another option. But for me it is important to have local
documentation and I would like to include building API pages in
"standard" build, which is problematic due to dependencies (I
have now dependencies on my main work machine, but until recently
I could run build of API pages only on remote machine) and due
to build time.

> You mentioned a few issues like to many <pre> stuff etc. or sidebars
> that use too much space. Well, yes, that can and should be improved, but
> that doesn't mean that we/you should *replace* the sphinx generated
> pages by server-side generated HTML pages. If you want to create such a
> system, do so, but do not ask for removing static pages (I somehow have
> the impression that such would be your long term goal -- if it is not
> then I apologise).

I hope that avoiding Sphinx will give at least as good and possibly
better result. Concerning API site, I am for keeping it as long
as is needed. If you think that Sphinx gives better result or
for whatever reason you want to use Sphinx for API site, that is
fine. I would like to avoid having needless duplication, and
have common code for all output formats, but intend to keep
Restructured Text as one of output formats.

> > > Would it be possible to retain https://fricas.github.io/ for what it
> > > does now and just link to the dynamic pages for things like:
> > > searching function names or live execution of examples.
> >
> > Yes.
>
> Yes, that sounds like a good approach.
>
> > > In practice I wonder how useful live execution of examples is?
> > > Its hard enough/impossible to get people to write even minimal
> > > documentation and this live execution would be a lot more work on
> > > top of that. So realistically, is it likely to increase beyond
> > > the small number of examples already in HyperDoc?
>
> > Not so small if you count FriCAS book.
>
> Eventually, we should also have the FriCAS book in HTML format on the
> web and maybe even with links to notebooks that one can download and
> execute.

Yes.

> > Concerning usefulness, I think that this is useful, but I want to get
> > other things working with HTML first, especially search.
>
> Can you be a bit more precise? Do you mean for example "search for
> exported functions in Foo(R) where a user can set R==Integer" or "search
> for the domain/package that implements function foo from domain Foo(R)
> where R is Fraction(Integer)"? Such things I would really appreciate and
> that wouldn't be a big security issue.

The two are assentially the same thing, and yes I would like to
prowide this. Potentially troublesome is someting like (untested):

ZMOD((systemCommand("system rm -rf /"); 3))

which (assuming I made no silly mistake) is supposed to "work" in
current HyperDoc and clearly I do not want to make accessible over
internet.

There are simpler things, that I think are valuable: per operation
views ("full" with descriptions, signatures, grouped by conditon),
wildcards in operations names, filtering on number of arguments,
etc. For constructors ancestor views, descendants, etc. That
is quite a lot of small pages, giving just requested info and no
more.

> > HyperDoc currently handles FriCAS book in C code (FRICASsys is only
> > involved in live examples). 'api.spad' contains convertor from LaTeX
> > susbset used in docstings to Restructured Text. IMO the approach will
> > not scale up to more complex tasks, so I want a different convertor.
>
> What are the "more complex tasks"?

FriCAS book.

> > I would like new convertor to handle FriCAS book.
>
> What exactly is your goal?

To be able to present FriCAS book as HTML/XML. Thanks to
MathML format it should be reasonably easy to get static
version of examples from FriCAS book in XML. With good
convertor we should be able to produce HTML version of
actual text.

> > > > Well, concerning API pages: negative part is that I do not see
> > > > how we can get good search using current technology (that is
> > > > Sphinx).
>
> The question is what you understand by "good search". I have not taken
> much effort to tailor the stuff that sphinx produces by default.
> I find that search field useful enough (not perfect) to get to the
> information I want.

For me API search produces long list of results which I would have
to manually scan to get what I need. HyperDoc at top allow to specify
if I want a constructor, an operation or possibly wider search.
Once I get search result, there are possibilities to choose a view
and/or further filter to result. This is documented in FriCAS
book and for me quite useful.

> Otherwise it might also make sense to just use
> google (as mentioned above).

Google is sometimes surprisingly good if you give wrong (say
misspeled) query, but may be poor for exact match or for
conditions it does not undersatand.

> And yes, search for specified parameters on
> parametrized domains would be super-cool and certainly nothing that
> sphinx can or should deliver.
>
>
> > > > Also, I find build time problematic. I would like to resolve
> > > > those problems, and I think that direct generation of HTML can
> > > > resolve both problems.
>
> I also think that the build time is too long. If someone can help in
> finding the reason for this, I'd appreciate. But removing sphinx would
> be a step into the wrong direction. No problem if you provide api2.spad
> generated html documentation on fricas.org.
>
> > > So I guess you want all the official documentation to be derived
> > > from some common root?
>
> > Yes, exactly.
>
> And for that it would be a good idea to agree upon a format that is both
> readable in source code (++ docstrings) and has enough information to
> generate good looking webpages from it. I'd really prefer reST over
> HyperTex.

Hmm, would you like to write DLMF or Wolfram functions site in
reST?

--
Waldek Hebisch

[Dima Pasechnik]

On Sun, Dec 15, 2024 at 7:22 PM Waldek Hebisch <de...@fricas.org> wrote:
>
> On Sat, Dec 14, 2024 at 02:38:35AM +0100, 'Ralf Hemmecke' via FriCAS - computer algebra system wrote:
> > On 12/14/24 00:00, Waldek Hebisch wrote:
> > > > > as static pages. Simply, instead of pr-generating inpractically
> > > > > large number of pages sever may generate them on the fly.
> >
> > Maybe, there are different people that have different goals...
> >
> > Perhaps it would be useful to clarify what each of us wants and perhaps
> > it makes more sense not to work against each other. To say the truth, at
> > the moment I feel as if you do not value the work that I put into
> > fricas.github.io and rather aim at creating your own way to generate
> > documentation and get rid of what I did. I hope you understand that this
> > wouldn't make me happy.
>
> Well, fricas.github.io is nice and I value the work you put into it.
> OTOH Sphinx IMO is a problematic dependency, so I would like to
> have equivalent effect without using Sphinx. Also, I want more
> functionality, some which looks impossible to get using Sphinx.

What's problematic about it, apart from a steep learning curve?

Sphinx can do a lot of things; e.g. if you don't like .rst, you can
use Markdown (.md);
you can use dynamic building - calling your system to generate output
and insert it into the docs you are building,
etc.
It's a de facto standard in the Python world, and very popular
documentation tool for many other environments, too.

Dima

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/fricas-devel/Z1-AtMqp7-VhVzci%40fricas.org.

[Waldek Hebisch]

On Mon, Dec 16, 2024 at 12:54:06PM -0600, Dima Pasechnik wrote:
> On Sun, Dec 15, 2024 at 7:22 PM Waldek Hebisch <de...@fricas.org> wrote:
> >
> > On Sat, Dec 14, 2024 at 02:38:35AM +0100, 'Ralf Hemmecke' via FriCAS - computer algebra system wrote:
> > > On 12/14/24 00:00, Waldek Hebisch wrote:
> > > > > > as static pages. Simply, instead of pr-generating inpractically
> > > > > > large number of pages sever may generate them on the fly.
> > >
> > > Maybe, there are different people that have different goals...
> > >
> > > Perhaps it would be useful to clarify what each of us wants and perhaps
> > > it makes more sense not to work against each other. To say the truth, at
> > > the moment I feel as if you do not value the work that I put into
> > > fricas.github.io and rather aim at creating your own way to generate
> > > documentation and get rid of what I did. I hope you understand that this
> > > wouldn't make me happy.
> >
> > Well, fricas.github.io is nice and I value the work you put into it.
> > OTOH Sphinx IMO is a problematic dependency, so I would like to
> > have equivalent effect without using Sphinx. Also, I want more
> > functionality, some which looks impossible to get using Sphinx.
>
> What's problematic about it, apart from a steep learning curve?

1) Build time
2) Python version dependencies
3) As used now it generated a fixed collection of pages, that
is problematic if we want to show information in many different
ways, in particular for showing search results.

> Sphinx can do a lot of things; e.g. if you don't like .rst, you can
> use Markdown (.md);
> you can use dynamic building - calling your system to generate output
> and insert it into the docs you are building,
> etc.
> It's a de facto standard in the Python world, and very popular
> documentation tool for many other environments, too.

If you are commited to Python (IIUC you are), you already payed
Python "cost", so you preceive it as no trouble. But, would you
use Sphinx to write something like DLMF or Wolfram function site
(at least part of our documentation preferably should have similar
form). I understand that Sphinx being Python code is preferable
to non-Python solution in Python world.

Currently my version of api2.spad is 467 lines, of which about
89 are trivial modification of Ralf's code and eventially should
be shared with code generating Restructured Text ('api2.spad'
also calls 'api.spad' so there is much more sharing, but ATM
those 89 lines need trivial HTML-specific modification, so
can not be called). So there is 378 lines of code specifically
to generate HTML. Compared to 'api.spad' this still does not
transform docstrings, but almost all other "logical" things
are there: conditions are handled now, sorting is reasonable
but need small tweak, I need to better escape special HTML
characters. So what remains is mainly transformation of
docstrings, which probably would need say 40 lines if done
in similar way as Ralf did transformation to Restructured
Text, but as I wrote, I would like to make more use of
transformation code so it almost surely will be bigger.

Another thing is having good HTML templates. Current template
is bare-bones and implicit (embedded as strings in Spad code).
Better templates potentially may go to some thousends lines,
we will see. But if one belives that Sphinx has/produces
best templates we could use Sphinx to generate templates once
and reuse them for different content. So I do not see HTML
templates as significant obstacle for direct HTML generation.
Simply, we can spent effort on templates if we think that
this gives "better" pages or with minimal effort we can reuse
existing templates.

So actual developement effort for API pages is likley to be
below 1000 lines of Spad code (I allow here substantial margin
for bug fixes and currently unimplemented part). Past experience
suggest that once written there will be small maintanence effort.

So one question, on API level: "is not writing 1000 lines of
Spad worth learning Sphinx, writing configuration files,
handling dependencies, etc.". IMO, no. And if we used
possiblity of adding hooks or extending Sphinx we easily
could end up adding similar amount of code.

Different projects may have different tradeofs. For example
API info is available and must be handled by Spad compiler.
So we have info available as nice data structures (except
for doctrings). Actually, current compiler data structures
are at Boot level, so there was work to expose/transform them
for use in Spad (done by Ralf). But the point is that data
structures are there. I suspect that in Python world, whatever
can be extracted from Python is handled by Sphinx, which
is quite differenet tradeoff.

> To view this discussion visit https://groups.google.com/d/msgid/fricas-devel/CAAWYfq2hE0sfs8FhR_2uLfLzpzDid4rV1kTrK-2x0s66a6QFjQ%40mail.gmail.com.

--
Waldek Hebisch

[Dima Pasechnik]

On Mon, Dec 16, 2024 at 6:04 PM Waldek Hebisch <de...@fricas.org> wrote:
>
> On Mon, Dec 16, 2024 at 12:54:06PM -0600, Dima Pasechnik wrote:
> > On Sun, Dec 15, 2024 at 7:22 PM Waldek Hebisch <de...@fricas.org> wrote:
> > >
> > > On Sat, Dec 14, 2024 at 02:38:35AM +0100, 'Ralf Hemmecke' via FriCAS - computer algebra system wrote:
> > > > On 12/14/24 00:00, Waldek Hebisch wrote:
> > > > > > > as static pages.  Simply, instead of pr-generating inpractically
> > > > > > > large number of pages sever may generate them on the fly.
> > > >
> > > > Maybe, there are different people that have different goals...
> > > >
> > > > Perhaps it would be useful to clarify what each of us wants and perhaps
> > > > it makes more sense not to work against each other. To say the truth, at
> > > > the moment I feel as if you do not value the work that I put into
> > > > fricas.github.io and rather aim at creating your own way to generate
> > > > documentation and get rid of what I did. I hope you understand that this
> > > > wouldn't make me happy.
> > >
> > > Well, fricas.github.io is nice and I value the work you put into it.
> > > OTOH Sphinx IMO is a problematic dependency, so I would like to
> > > have equivalent effect without using Sphinx.  Also, I want more
> > > functionality, some which looks impossible to get using Sphinx.
> >
> > What's problematic about it, apart from a steep learning curve?
>

> 1) Build time

incremental builds are fast, so if you work on changing docs it's not really

a log wait.

> 2) Python version dependencies

You aren't really writing Python code. Thus, it's just

python3 -m venv foo

source foo/bin/activate

pip install sphinx

> 3) As used now it generated a fixed collection of pages, that
>    is problematic if we want to show information in many different

>    ways, in particular for showing search results.

you search in this fixed collection, how is it problematic?

In particular, if you want to get searches from the internet, how else can you do it

without a live server running FriCAS?

>
> > Sphinx can do a lot of things; e.g. if you don't like .rst, you can
> > use Markdown (.md);
> > you can use dynamic building - calling your system to generate output
> > and insert it into the docs you are building,
> > etc.
> > It's a de facto standard in the Python world, and very popular
> > documentation tool for  many other environments, too.
>
> If you are commited to Python (IIUC you are), you already payed

> Python "cost", so you preceive it as no trouble. 

This can be done by someone without much understanding of FriCAS,

that's why Sphinx is a good thing.

> To view this discussion visit https://groups.google.com/d/msgid/fricas-devel/Z2DAB7_iqwqP4KYz%40fricas.org.

[Ralf Hemmecke]

>> So one question, on API level: "is not writing 1000 lines of
>> Spad worth learning Sphinx, writing configuration files,
>> handling dependencies, etc.". IMO, no. And if we used
>> possiblity of adding hooks or extending Sphinx we easily
>> could end up adding similar amount of code.
>
> This can be done by someone without much understanding of FriCAS,
> that's why Sphinx is a good thing.

Standing on the shoulders of giants ... vs. reinventing the wheel...

Hi Dima,

>> 1) Build time
> incremental builds are fast, so if you work on changing docs it's not really
> a log wait.

Can you share the commands to achieve incremental builds?

Currently the problem certainly is that all .rst files for the API are
overridden so if 'make html' checks the time stamps then I would have to
take more care not to override identical .rst files.

Do you happen to have any idea how I can simply generate
<code>reduce(f, a, r)</code> blocks from ``reduce(f, a, r)``
instead of

<code class="code docutils literal notranslate"><span
class="pre">reduce(f,</span> <span class="pre">a,</span> <span

class="pre">r)</span></code>

My suspicion is that it is connected to pygments (although I have

highlight_language = 'none'

set in conf.py) or perhaps I could set another HTML generator for code
blocs. Unfortunately I've not yet found where and how. :-(
Do you have any hints/links?

Thanks

Ralf

[Kurt Pagani]

As a big fan of the current API docs I think it should be possible to speed up the sphinx build ("sphinx speed up build"@duckduckgo gives a lot of hints):

https://stackoverflow.com/questions/51986357/how-can-i-make-sphinx-build-build-faster

https://get-started-with-sphinx-and-rst.readthedocs.io/improve-building/

https://frankwiles.com/posts/automatically-rebuild-sphinx-docs/

Of course, nobody will hardly object to alternatives, however, for me and maybe others  it's practical and indispensable. To drop a clue: coexistence ;)

[Waldek Hebisch]

On Mon, Dec 02, 2024 at 12:38:14AM +0100, Waldek Hebisch wrote:
> I have tried to modify Ralf's code
> to directly generate HTML, skipping the .rst stage. You
> can see proof of concept code at:
>
> http://wiki.fricas.org/public/api2.spad

At the same place there is now new, better version. Docstrings,
condition and navgation links to defining place are there.
I added also some escaping of special HTML characters.
Even without escaping the characters appeared fine in Lynx,
but possibly without escaping other browsers could fail or
show wrong thing.

Checking of docstrings is quite picky, it will signal error
on anything unexpected. In a sense main addition is hopefully
reasonably general parser for TeX/LaTeX files. However, it
has table of known TeX macros an currently reject any unknown
macro. It the trunk I have corrected errors and irregulrities
discoverd by docstring checks, so it should work with the trunk.
But older version are likely to fail.

This version has much less formatting HTML and navigation
links than output from sphinx+api.spad (on my machine total
size from sphinx+api.spad is 45834587 characters while size
from api2.spad is 15390243). However, adding extra stuff now
should be much easier than what is already done.

> If you want to try this, first generate 'abbrev-lines.list'
> (you can take it from documentation build), then compile
> 'api.spad' from 'src/doc', then compile 'api2.spad' and
> at FriCAS prompt issue
>
> documentAll("abbrev-lines.list")

The same with this version.

--
Waldek Hebisch

[Kurt Pagani]

I just tried it. Working well up to:

 

...

 "ComplexRootPackage"

   "ComplexTrigonometricManipulations"
   "compUtil"
   \x
 
   >> Error detected within library code:
   parse_macro: unknown macro name

The html (source) output so far is quite terse, no doubt.

BTW, "compUtil.html" has been written but is empty.

[Waldek Hebisch]

On Wed, Dec 25, 2024 at 04:46:54AM -0800, Kurt Pagani wrote:
> I just tried it. Working well up to:
>
> ...
> "ComplexRootPackage"
> "ComplexTrigonometricManipulations"
> "compUtil"
> \x
>
> >> Error detected within library code:
> parse_macro: unknown macro name
>
> The html (source) output so far is quite terse, no doubt.
> BTW, "compUtil.html" has been written but is empty.

This is one of irregularities discovered. The '\x' is no longer
present in the trunk.

> On Wednesday, 25 December 2024 at 05:58:06 UTC+1 Waldek Hebisch wrote:
>
> > On Mon, Dec 02, 2024 at 12:38:14AM +0100, Waldek Hebisch wrote:
> > > I have tried to modify Ralf's code
> > > to directly generate HTML, skipping the .rst stage. You
> > > can see proof of concept code at:
> > >
> > > http://wiki.fricas.org/public/api2.spad
> >
> > At the same place there is now new, better version. Docstrings,
> > condition and navgation links to defining place are there.
> > I added also some escaping of special HTML characters.
> > Even without escaping the characters appeared fine in Lynx,
> > but possibly without escaping other browsers could fail or
> > show wrong thing.
> >
> > Checking of docstrings is quite picky, it will signal error
> > on anything unexpected. In a sense main addition is hopefully
> > reasonably general parser for TeX/LaTeX files. However, it
> > has table of known TeX macros an currently reject any unknown
> > macro. It the trunk I have corrected errors and irregulrities
> > discoverd by docstring checks, so it should work with the trunk.
> > But older version are likely to fail.

^^^^^^^^^^^^^^

As happened in your case.
<snip>

> \x
>
> >> Error detected within library code:
> parse_macro: unknown macro name
>

> (6) -> )version
> "FriCAS 2024-04-15 compiled at Wed Apr 24 04:36:24 PM CEST 2024"
> (6) ->

Trunk now shows 'FriCAS 2024-12-20'. And you need commit
fd8a0347e0684a750bdad25d8188580861bbd406

Alternatively, exclude problematic constructors like

compUtil
BasicKeyedAccessFile
NumericalOrdinaryDifferentialEquations

from abbrev-lines.list. I am not sure if this is complete list
of files that will give error, there are other changes that I
made to correct wrong output. Possibly without corrections
errors could appear in files that were later changed.

--
Waldek Hebisch

[Kurt Pagani]

Indeed, I thought twice whether to report before upgrading.

No issues now with the new list.

...

   "XPolynomialFactor"
   "ZeroDimensionalSolvePackage"
                                                                   Type: Void
(7) -> )version

"FriCAS 2024-12-20 compiled at Wed 25 Dec 17:23:28 CET 2024"

[Kurt Pagani]

I took the liberty to upload the output temporarily to https://nilqed.github.io/api2/ for better reviewing.

If you would insert a line like (in or before <head>)

   <link rel="stylesheet" href="styles.css">

in each html file, then everyone could change the appearance with just one css file.

I made an example (no aesthetic claims, just for demo ;-):

https://nilqed.github.io/api2/AlgebraGivenByStructuralConstants.html

with styles.css:

https://nilqed.github.io/api2/styles.css

In lynx, w3m or links it doesn't really makes a difference, however, when viewing online (e.g. firefox) then it is IMO a relief when the underlined links are gone ...

[Waldek Hebisch]

On Thu, Dec 26, 2024 at 03:01:58AM -0800, Kurt Pagani wrote:
> I took the liberty to upload the output temporarily to
> https://nilqed.github.io/api2/ for better reviewing.
> If you would insert a line like (in or before <head>)
> <link rel="stylesheet" href="styles.css">
> in each html file, then everyone could change the appearance with just one
> css file.

Sure, I have put modified "api2.spad" at the same place, that is

http://wiki.fricas.org/public/api2.spad

> I made an example (no aesthetic claims, just for demo ;-):
> https://nilqed.github.io/api2/AlgebraGivenByStructuralConstants.html
> with styles.css:
> https://nilqed.github.io/api2/styles.css
>
> In lynx, w3m or links it doesn't really makes a difference, however, when
> viewing online (e.g. firefox) then it is IMO a relief when the underlined
> links are gone ...

Yes, I looked at what we need:

- title ???
- header
- <code> blocks
- references to constructors and functions
- function/operator names

Maybe we want to distinguish text from doctrings (but almost all
actual text is from doctrings) or conditions on operations/categories.

AFAICS Sphinx style uses changed font (bigger and different shape) and
colour for header, changed font (look like sanserif font) and colour
for references to constructors and functions, light gray colour
(and possibly changed font) for <code> blocks and changed font
(also sanserif) and colour (red) for function/operator names.

--
Waldek Hebisch

[Kurt Pagani]

On Thursday, 26 December 2024 at 14:44:56 UTC+1 Waldek Hebisch wrote:

On Thu, Dec 26, 2024 at 03:01:58AM -0800, Kurt Pagani wrote:
> I took the liberty to upload the output temporarily to
> https://nilqed.github.io/api2/ for better reviewing.
> If you would insert a line like (in or before <head>)
> <link rel="stylesheet" href="styles.css">
> in each html file, then everyone could change the appearance with just one
> css file.

Sure, I have put modified "api2.spad" at the same place, that is

http://wiki.fricas.org/public/api2.spad

Thank you, works fine.

 

> I made an example (no aesthetic claims, just for demo ;-):
> https://nilqed.github.io/api2/AlgebraGivenByStructuralConstants.html
> with styles.css:
> https://nilqed.github.io/api2/styles.css
>
> In lynx, w3m or links it doesn't really makes a difference, however, when
> viewing online (e.g. firefox) then it is IMO a relief when the underlined
> links are gone ...

Yes, I looked at what we need:

- title ???
- header
- <code> blocks
- references to constructors and functions
- function/operator names

Maybe we want to distinguish text from doctrings (but almost all
actual text is from doctrings) or conditions on operations/categories.

Now, no or an empty "styles.css" will display default html. Who wants to change the appearance can now do so by editing the "styles.css".

Since I have only rudimentary knowledge about CSS (from https://www.w3schools.com/Css/), I couldn't say what is possible ...  

 

Now, no or an empty "styles.css" will display default html.

AFAICS Sphinx style uses changed font (bigger and different shape) and
colour for header, changed font (look like sanserif font) and colour
for references to constructors and functions, light gray colour
(and possibly changed font) for <code> blocks and changed font
(also sanserif) and colour (red) for function/operator names.

https://www.sphinx-doc.org/en/master/usage/theming.html

There are so many sphinx themes -- https://sphinx-themes.org/ -- really head spinning.

 

--
Waldek Hebisch

[Waldek Hebisch]

On Mon, Dec 02, 2024 at 12:38:14AM +0100, Waldek Hebisch wrote:

> I have tried to modify Ralf's code
> to directly generate HTML, skipping the .rst stage. You
> can see proof of concept code at:
>
> http://wiki.fricas.org/public/api2.spad

At the link above there now next, slightly modified version.
Also at:

http://wiki.fricas.org/public/styles.css

is my attempt at a stylesheet. There seem to be some discrepancies,
but IMO it reasonably well approximates colours and fonts of
Sphinx style used by Ralf.

Of course, we may try different style. For example by default
Firefox shows visited link in a different colour. I find this
useful, but Sphinx style apparantly want the same colour for
all links, visited or not.

> If you want to try this, first generate 'abbrev-lines.list'
> (you can take it from documentation build), then compile
> 'api.spad' from 'src/doc', then compile 'api2.spad' and
> at FriCAS prompt issue
>
> documentAll("abbrev-lines.list")

--
Waldek Hebisch

[Kurt Pagani]

On Friday, 27 December 2024 at 03:32:24 UTC+1 Waldek Hebisch wrote:

On Mon, Dec 02, 2024 at 12:38:14AM +0100, Waldek Hebisch wrote:
> I have tried to modify Ralf's code
> to directly generate HTML, skipping the .rst stage. You
> can see proof of concept code at:
>
> http://wiki.fricas.org/public/api2.spad

At the link above there now next, slightly modified version.
Also at:

http://wiki.fricas.org/public/styles.css

is my attempt at a stylesheet. There seem to be some discrepancies,
but IMO it reasonably well approximates colours and fonts of
Sphinx style used by Ralf.

I pushed it and it comes quite close , indeed.

Of course, we may try different style. For example by default
Firefox shows visited link in a different colour. I find this
useful, but Sphinx style apparantly want the same colour for
all links, visited or not.

Well, I'm already quite happy with this ... but will be probably be a lcd in the end.

What's also quite apparent is the improvement on the speed of displaying - presumably the absence of javascript !?