Ann: Shen; the Open Science Initiative

110 views
Skip to first unread message

dr.mt...@gmail.com

unread,
Sep 4, 2026, 8:58:18 AMSep 4
to Shen
A lot of changes are coming to the Shen site and they will happen
systematically over the next few months.   Let me detail the changes
in order.

1.  Upgrading the Shen site.   This has already begun
with getting the zoom right.  It is actually IMO a pretty nice design.
2.  Getting rid of dead links.  We have a few, a lot of links to external 
sites which no longer exist.   I'm closing on these and 50 dead links
are now down to 34.
3.  Simplifying downloads.  All the Shen apps are now in Lib along
with the kernel sources and are under BSD.   So when you download
the zipped kernel sources, you download everything else.
4.  Dumping outdated information.   The apps are cool but sometimes
the docs are out of date in parts.  So systematically I'm going to go
through the docs and make sure they fit what is actually there.
5.  Switching to pdf.   This means TBoS will appear in pdf and hardcopy
along with everything else.  People definitely want this.   It also makes
the site smaller wrt page count and easier to maintain.  Of course
'smaller' here does not mean less info!
6.  Formal Proofs  There is a lot of crafty work in Shen, like the algorithm
for factorising code that works.  But in some cases the correctness proof
is missing.   I want these proofs in place and when done the result will be
publishable.  This brings me to the next point.
7.  Conference appearances and publication   This will be a focus.  I
already had a success in having a Shen paper accepted for Intellisys.
Conference appearances raise profile and the papers link Shen to
the scientific community and make it respectable for people to use
it in research and cite it.   It will raise our membership profile.

However I backed out of Intellisys because I was expected to pay £750.   I'm 
going to try to limit the costs as much as I can but really I cannot
be expected to pay out of my own pocket for making my work free.   The 
volume of publishable material in Shen would cost more than I can
afford.

So I will be asking the group to donate when a paper gets accepted.
I'll have a drive to do that and I'll announce it.  

This is part of the Open Science Initiative which I planned long ago
(2024 to be exact).   The relevant page is here.   This will all happen
systematically over the next 6 months and will be announced in 
Latest News.

Mark

dr.mt...@gmail.com

unread,
Sep 4, 2026, 11:36:02 AMSep 4
to Shen

2.  Getting rid of dead links.  We have a few, a lot of links to external 
sites which no longer exist.   I'm closing on these and 50 dead links
are now down to 34.

Down to 6.

M. 

dr.mt...@gmail.com

unread,
Sep 5, 2026, 3:08:21 PMSep 5
to Shen
I'm now recoding the site generator.  This will be
an order of magnitude smaller than the hastily
hacked version running at the moment, and
will generate HTML with 40% of the footprint
of the current version.  I've converted TBoS
to txt and and testing a page generator indexed
by sections.  

If this works we will be able to feed in books
from Word and get an indexed HTML directory.
Word does convert to HTML, but the result is
not great nor suitable for large documents.

ChatGPT brilliantly swapped unicode for
the non-ASCII characters in txt TBoS which
is great.

M.

dr.mt...@gmail.com

unread,
Sep 6, 2026, 3:25:40 PMSep 6
to Shen
On Saturday, 5 September 2026 at 20:08:21 UTC+1 dr.mt...@gmail.com wrote:
I'm now recoding the site generator.  This will be
an order of magnitude smaller than the hastily
hacked version running at the moment, and
will generate HTML with 40% of the footprint
of the current version.  I've converted TBoS
to txt and tested a page generator indexed
by sections.  

If this works we will be able to feed in books
from Word and get an indexed HTML directory.
Word does convert to HTML, but the result is
not great nor suitable for large documents.

Sadly it does not.  Conversion to txt loses too
much formatting which can only be restored by AI
- and even then this is spotty.  So we're back to 
embedded jpg.  However I've greatly increased
the dpi this time, so the result is very crisp.

The generator is down from 1800 loc to 260 loc
the result of placing style declarations up front instead
of repeating them.  As a result the HTML is cut
in half.   Also I've factorised the code for generating
web pages down to one elegant program.

A lot of space in the old generator was taken up in 
generating special pages which I've not yet done
in the new one.  

FYI here is the code for the Shen community page.

{style.page1 Community \
 
  {style.p There are over 500 members in the Shen news group to approach for questions.  We also have the 2011 committee
  who exist to advise on the development of Shen.  As of 2021 these are:}
 
  {style.table Dr Mark Tarver | Chairman of the committee. Graduate of the Universities of Reading, Oxford, Warwick
                                and Leeds, Dr Tarver is the originator of Shen and the current license holder. He built
                                the first Common Lisp port for Shen. ||
               Ramil Farkhshatov | Ramil built the Javascript and Python ports for Shen. ||
               Bruno Deferrari   | Bruno built and maintains the Scheme port of Shen. He also hosts the
                                    github mirror for OS Shen releases. ||
               Mark Thom         | Mark built and maintains the Haskell port of Shen. ||
               Tatsuya Tsuda     | Tatsuya built and maintains the C port of Shen. He also built the iOS app of Shen. ||
               Chris Double       |  Chris ported Shen to Wasp Lisp ||  
               Neal Alexander     |  Neal was the prime driver in the development of Shen Professional. ||}              
 
{style.p We'd like to recognise the following emeritus former members of the committee who served their time and made
significant contributions to Shen; Vasil Diadov, Kian Wilcox, Hakan Raberg, Carl Shapiro, Aditya Siram, Willi Riha,
Greg Spurrier. }          
               
  {style.p Some community resources ...}
     
{style.table  {style.a https://groups.google.com/forum/?hl=en#!forum/qilang \ Online News Group}  | the place for all your questions ||
              {style.a chatroom.html \ IRC chat channel} | chat interactively on Shen ||
              {style.a https://www.reddit.com/r/shenlanguage \ Shen on Reddit} | for reddit users ||} \
{style.sidelinks}}

This is a sort of DSL based on smartstring technology which is found in the Standard Library.

Mark

Reuben

unread,
Sep 8, 2026, 3:35:32 PMSep 8
to Shen
I think with a combination of image -> pdf parsing, AI image recognition, and text parsing of the txt version, we could probably build an AI pipeline that converts to actual HTML. 

https://grok.com/share/bGVnYWN5_7980abe4-72a7-4d0e-a03d-e7ae5f01a413

dr.mt...@gmail.com

unread,
Sep 8, 2026, 3:49:21 PMSep 8
to Shen
I think it could be done but it might not be algorithmic
in the sense generally understood.  Bringing AI into the
pipeline makes the whole process of site generation very
interactive and context sensitive (which AI? and who is
training it?).  The current method is totally algorithmic
and generates 788 web pages in 420s - thats about 70% of
the final size of the site.  

AI is great but if you are requiring intelligence and
inspiration to do something then you lose out on predictability
which is one reason we love computers.  

M.

Mark Tarver

unread,
Sep 8, 2026, 4:57:36 PMSep 8
to qil...@googlegroups.com
One thing which is interesting is that when you build your
own site generator there is a bit of a question about what is style and
what is content.   The reason my style file is much shorter  (but still now 484 loc)
is that I decided that what was previously style could be put down as content 
and moved to the site pages.  The other reason was grasping the utility
of placing style declarations in one place.   

M.

--
You received this message because you are subscribed to a topic in the Google Groups "Shen" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qilang/n3B5EWS2vSU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qilang+un...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/qilang/5f329b0b-4176-47bc-b0e0-aefe5aac524an%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages