some thoughts on working with LLMs on the S series revival program

[dr.mt...@gmail.com]

Well gentlemen, it's a revolution in working, no doubt about it.  Something that

would be thought impossible for one person was done in less than a week;  

reviving 4 defunct ports.  What was it like and what did I learn and where does

all this go?

Well before we get into that you can get the gen on the project from here

and from the download page which gives the S-series revival downloads for 

Python, Javascript, Java and Ruby ports for Shen.  All of the ports have

passed the 134 tests that constitute certification.

What was it Like?

 Not like programming for sure.   Compared to the belly crawl over every

square metre that is programming, this was more like flying in Concorde

at 40,000 feet.   Being served canapes and champagne in first class.

Did I understand what was going on?

Not really.  And yet to undermine my analogy, I was not a passenger.

Several times human intuition from my side pointed a way forward

because architectual understanding can be language independent.

But most of my time was spent unzipping downloads,  relaying errors

and recounting timings.   Our first Python revival took 750 sec to

complete the test suite.  The final version 111 sec.

How do humans compare?

Interesting question.  I was left with a high degree of respect for

what the human porters had done.  They had done the heavy lifting.

In the Python case ChatGPT got creative and this resulted in 

mysterious call stack overflows until I told it to stop buggering 

around and follow Matthieu Lagacherie and Yannick Drant, who

did the original work.   After that things went fine.

It was clear that ChatGPT understood very quickly that the 

S kernel made porting much easier than the pre-S kernels and

that a lot of cruft wrt partial applications could be thrown away.  

Hence the work really consisted of throwing away this cruft and 

dropping obsolete tooling from Python et al.  So there was a clear 

pattern, and the more ports we revived, the better ChatGPT got at doing it.

How easy is it to maintain the revived ports?

This is another fascinating question.   Very easy in a sense and very

hard in another. Very easy because I insisted on a uniform approach

to rebuilding the ports.   Each port builds from the KLambda directory

and so swapping this out for an upgrade is no harder than replacing the 

batteries in your torch.   But the price is that I do not understand what

is under the hood and correctness has to be maintained by testing.

Are humans obsolete?

No.  A certain way of doing things is obsolete.   But ChatGPT is a

short range weapon that likes highly structured tasks.  Point it at

a well defined problem with plenty of previous work and its like a 

buzz saw.   Ask too much and it circles needing guidance.  For example,

ChatGPT said that Tatsuya Tsuda's C port was an interpreter (I guess

rather like Trealla Prolog).   So I asked 'Could you build a compiled

version'.  It said it could but could not.  Large unguided projects are too

much for it.

Where do we go from here?

Well I have a cunning plan, Baldrick style.  But I'm not reviving projects

which are active in Github.  I see just now that the list of active projects

on https://shen-language.github.io/ has suddenly burgeoned from 

me and Bruno to a whole list and that has to be good.   I'll come back

to those projects some other time.

So basically, to all those who built those ports; nothing is lost.  

Your work will continue to carry significance and you will live on through

it, and that is as much as one could wish for.

Mark

[Bruno Deferrari]

On Thu, Sep 17, 2026 at 8:13 PM dr.mt...@gmail.com <dr.mt...@gmail.com> wrote:

I see just now that the list of active projects

on https://shen-language.github.io/ has suddenly burgeoned from 

me and Bruno to a whole list and that has to be good.   I'll come back

to those projects some other time.

Credit for the expanded active list on that page goes to Reuben who revived some of the existing ports, and created new ones too.

--

BD

[Reuben]

Your observations on using AI to maintain the ports ring very true. Particularly the point about testing, as well as the quality of the work done in the original ports.

I found it very helpful to use the ports to benchmark each other, as well as to provide context / ideas to the LLM for how a different language implemented XYZ problem, and to consider whether or not that might be applicable in a different language port. This often led to much better-performing implementations than just "how could this run faster" or "pass all the tests."

This cross-referencing was a big part of why I put github.com/pyrex41/bifrost together -- I wanted a simple way to easily run the same tests or benchmarks on multiple ports. This has allowed me to easily use Shen for different kinds of projects, and also uncovered multiple bugs that were not originally covered in the various test suites.

[Reuben]

I also did some work on Tatsuya Tsuda's -- and also similarly concluded that a fully compiled version was not possible without tremendous effort, and even then, you really couldn't get away from some kind of garbage collection.

However, leveraging Yggdrasil, I was able to create a kind of hybrid that would compile at least some subsets of Shen programs. ChatGPT analysis here

https://github.com/otabat/shen-c

On Thursday, September 17, 2026 at 7:06:08 PM UTC-5 Bruno Deferrari wrote:

[Reuben]

Correction -- I linked to the original. My fork is here:

https://github.com/pyrex41/shen-c

[dr.mt...@gmail.com]

|  I found it very helpful to use the ports to benchmark each other  .....

Sounds as if you put a lot of thought into it.

| I also did some work on Tatsuya Tsuda's -- and also similarly concluded that a fully compiled version was not possible without 

 | tremendous effort, and even then, you really couldn't get away from some kind of garbage collection.

Yes; same here.  The first attempt wo GC ran up a 5Gb memory debt and shut down my

computer.   I observed very recently your active work on Tsuda and declined to put this C port up.  We

(ChatGPT and I) got a version running and I'll now put it up anyway - it may have value.  The 

'tremendous effort' of a native C Shen IMO is beyond unguided ChatGPT.  

| This has allowed me to easily use Shen for different kinds of projects, and also uncovered 

| multiple bugs that were not originally covered in the various test suites.

The test suites were ad hoc; and really they were just a compendium of stuff written at that time.

However they do test YACC, Prolog and Shen.  But they are not systematically designed to handle

corner cases etc.  Greg Spurrier was very keen on mechanical testing in this area.

The times taken to execute the test suite in the revival are interesting

Python 111s
C      64s
Ruby   37s
SBCL   3s
Java   7.3s
Javascript 59s
Scheme  1s

The bolded entries are not revival ports but active human ports.  They are faster partly because

of the platforms underneath.  Wrt the other times I do not know the speed of the originals or

whether ChatGPT missed out on original optimisations.  

It looks as if you're using LLM techniques to revive ports much as I am.  So basically the revival

is being driven from two ends.  It's good.  I can't point to your forks w.o. knowing that they are

working and which ones.   It would be good to stabilise on a shared minimal directory architecture to

this extent.  A clear installation process with the KLambda directory at the top level to allow new

kernels to be swapped in with minimum fuss.  

Mark

[Bruno Deferrari]

On Fri, Sep 18, 2026 at 12:49 PM Reuben <reub....@gmail.com> wrote:

I also did some work on Tatsuya Tsuda's -- and also similarly concluded that a fully compiled version was not possible without tremendous effort, and even then, you really couldn't get away from some kind of garbage collection.

It would be a tremendous effort if done manually, but a properly guided agent can be incredibly effective at it. 

For the GC you can use something existing like Boehm, or just point the agent to good GC implementations (OCaml and Chez are very good references).

You also need to be explicit about good memory representations for values, since this will drive everything. 

You don't need to write a spec or anything, you can use the agent for that too, but you need to be explicit about it.

If you are going to make a custom GC, make sure that the agent uses relevant tools to make its (debugging) work easier (ASan and UBSan are a must if you are working with memory). 

Anyway, there are many things, you just have to do it with intention, throwing the problem at the agent and expecting magic is not enough (at least not with today's models).

I am working on exactly that btw: an implementation that produces native code directly (with plans for WASM too), and it already beats Shen/Scheme's speed at almost everything (Prolog is still a bit slower but it will be faster once I get to it). All code is being written by Codex, but with lots of guiding and guardrails to make sure it gets to where I want.

So far the results look good, but the code is a mess and I will throw it away once I am happy with the results and have produced the artifacts I need to be able to produce a clean rewrite and bootstrap it. Once the cleaned up version is complete I will publish it (name suggestions welcome btw).

 

However, leveraging Yggdrasil, I was able to create a kind of hybrid that would compile at least some subsets of Shen programs. ChatGPT analysis here

https://github.com/otabat/shen-c

On Thursday, September 17, 2026 at 7:06:08 PM UTC-5 Bruno Deferrari wrote:

On Thu, Sep 17, 2026 at 8:13 PM dr.mt...@gmail.com <dr.mt...@gmail.com> wrote:

I see just now that the list of active projects

on https://shen-language.github.io/ has suddenly burgeoned from 

me and Bruno to a whole list and that has to be good.   I'll come back

to those projects some other time.

Credit for the expanded active list on that page goes to Reuben who revived some of the existing ports, and created new ones too.

--

BD

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/qilang/d669c321-7443-40d7-85f5-d7dce09373f0n%40googlegroups.com.

--

BD

[Reuben]

The shen-rust port i did may be of interest. It's messy, but I got performance approaching sbcl in certain configurations. It also explores a number of different approaches achieve performance: https://github.com/pyrex41/shen-rust/blob/main/PERFORMANCE.md.

I leaned a lot on the shen-go and shen-c and sbcl ports for architecture ideas. I don't have deep knowledge in rust, I have to think someone who does could point the LLM to uncover better implementation choices. I was able to squeeze a lot out though.

Re: shared directory structure -- that's a great idea. I envision bifrost or something similar to make this super clean / easy -- here's an idea what could look like: https://github.com/pyrex41/bifrost/blob/main/docs/port-layout.md

@Bruno -- that is super cool. I've been leveraging my shen-go fork a lot because having go compile down into a go binary is just so nice for deployment in so many contexts, but a pure native compile target would be even better. I was able to get shen-rust to compile to wasm I think -- I can't remember how well that worked offhand.

[Bruno Deferrari]

On Fri, Sep 18, 2026 at 7:15 PM Reuben <reub....@gmail.com> wrote:

The shen-rust port i did may be of interest. It's messy, but I got performance approaching sbcl in certain configurations. It also explores a number of different approaches achieve performance: https://github.com/pyrex41/shen-rust/blob/main/PERFORMANCE.md.

I leaned a lot on the shen-go and shen-c and sbcl ports for architecture ideas. I don't have deep knowledge in rust, I have to think someone who does could point the LLM to uncover better implementation choices. I was able to squeeze a lot out though.

Point your agent at Shen/Scheme too. It is fast because Chez is fast, but also because the Shen->Scheme compiler does the necessary work to be able to exploit it well. 

The SBCL port has the same performance potential (at some point when I was maintaining it using a modified version of the Shen/Scheme compiler,

and its performance then was quite close to Shen/Scheme's).

It doesn't do anything complicated, it is mainly about taking advantage of KLambda properties and well placed overrides to kernel functions,

in a way that maps well to what the Chez compiler can handle effectively.

The Shen kernel provides inefficient implementations for many functions that are used a lot (for a good reason, these implementations are portable, 

and if not done like this, implementing a Shen port would be much harder), so other than the compiler itself generating good code for the underlying platform, 

overriding those functions is a very effective way to get better performance.

 

To view this discussion, visit https://groups.google.com/d/msgid/qilang/391f134e-78da-4685-a0a6-50185d01e91dn%40googlegroups.com.

--

BD

[dr.mt...@gmail.com]

I've put the revived C port up on the download page.

M.

[dr.mt...@gmail.com]

The Shen kernel provides inefficient implementations for many functions that are used a lot (for a good reason, these implementations are portable, and if not done like this, implementing a Shen port would be much harder), so other than the compiler itself generating good code for the underlying platform, overriding those functions is a very effective way to get better performance.

Actually that is exactly what I did with the CL port of pre-S kernels up to 2015.  Mainly to speed up the Prolog.

M.

[Bruno Deferrari]

On Fri, Sep 18, 2026 at 7:38 PM Bruno Deferrari <uti...@gmail.com> wrote:

On Fri, Sep 18, 2026 at 7:15 PM Reuben <reub....@gmail.com> wrote:

The shen-rust port i did may be of interest. It's messy, but I got performance approaching sbcl in certain configurations. It also explores a number of different approaches achieve performance: https://github.com/pyrex41/shen-rust/blob/main/PERFORMANCE.md.

I leaned a lot on the shen-go and shen-c and sbcl ports for architecture ideas. I don't have deep knowledge in rust, I have to think someone who does could point the LLM to uncover better implementation choices. I was able to squeeze a lot out though.

Point your agent at Shen/Scheme too.

Now there is a better reference (generated this from notes I had lying around, and what Shen/Scheme does when compiling the code), should be a better input for agents (and humans):

https://github.com/Shen-Language/shen-sources/blob/master/doc/port-performance.md

It includes current and historical optimizations (some that Shen/Scheme had to implement in the past, but that are not required anymore because of improvements to the Chez compiler itself) because historical ones may still be useful to other ports.

I will do some refinement passes later, but the current version should be useful already.

--

BD