Feature request: shen->kl

[Greg Spurrier]

Hi Mark,

Would it be possible to add a shen->kl function to the Shen system functions in an upcoming release?

I'm working on a new ShenRuby compiler implemented in Shen and this would help with bootstrapping it into a full system. Looking at the implementation of eval in sys.shen, it looks like this should be pretty straightforward. I'd be happy to do the work if you'll tell me how to generate Shen's K Lambda files from its Shen sources, or point me to documentation for that process.

Thanks,

Greg

[Mark Tarver]

Why not just write shen.shen->kl?

Mark

[deech]

Hi Mark and Greg,
I have requested this feature before and I'd like to do so again.

I attempted to generate my own KL files some time ago and didn't really have time to get it working all the way. I have included the scripts I wrote in the hopes that:
1. it might be useful to you and I hope you can get it working
2. to gently remind Dr. Tarver that this is a greatly missed feature that would allow the user to better understand the language implementation and submit patches.

Some notes on my scripts:
1. The function that does all the work is 'bootstrap.to-kl'. There was a reason that I couldn't use shen.shen->kl, but I can't remember what it is right now.
2. All the file IO is done using SBCL, so you'll probably have to port that to Ruby
3. One of stumbling blocks is that Shen will refuse to re-read system functions (Shen Sources/core.shen, line 70) so before reading a file I had to remove symbols that invoke the error and re-insert after I was done. The *clashing-symbols* variable lists the ones which cause the errors.
4. I got it working for most of the files, but it failed on 'sequent.shen' when I temporarily removed '@s' from 'external-symbols' since it was actually used in the file I was trying to parse. At this point I moved on.

Thanks!
-deech

[Greg Spurrier]

I'm fine with it being shen.shen->kl rather than a public system function, but would prefer that it be maintained as part of the official Shen sources. Otherwise, with each new Shen release, there is a risk of it breaking if the internal functions on which is depends change.

What I had in mind was refactoring eval and eval-without-macros so that they are implemented in terms of shen.shen->kl. I'd be happy to do that work if you'd like and will help me figure out how to generate Shen's K Lambda files from its Shen sources.

In the mean time, I'll write my own shen.shen->kl function that I can use to make progress.

Thanks,

Greg

[Mark Tarver]

[Mark Tarver]

On Jun 13, 1:46 pm, Greg Spurrier <greg.spurr...@gmail.com> wrote:

> I'm fine with it being shen.shen->kl rather than a public system function,
> but would prefer that it be maintained as part of the official Shen
> sources. Otherwise, with each new Shen release, there is a risk of it
> breaking if the internal functions on which is depends change.

I understand this to mean that you are concerned that the
functionality of 'shen->kl' may change from version X to version Y and
hence, because you want to use it in porting, you want to draw it into
the standard to prevent this. This is OK.

Regarding patches and the like to kernel code; if there is a fault I'd
rather have a bug report than a patch. Maintaining Shen across 9
different platforms and 3 OS is very difficult w.o. centralisation of
code.

Mark

[Greg Spurrier]

I understand this to mean that you are concerned that the
functionality of 'shen->kl' may change from version X to version Y and
hence, because you want to use it in porting, you want to draw it into
the standard to prevent this.  This is OK.

Correct. I would like to be able to depend on it being present and having consistent behavior from release to release.

Thanks,

Greg

[Mark Tarver]

I don't have a problem with what you want although I don't think the
issue you've raised is that central.  The functionality of shen->kl
has not changed since 2011 and you're not likely to go far wrong in
writing 'shen.shen->kl'.

Regarding refactoring the Shen kernel, of course one can do this
within the terms of the license as long as the license remains and the
result still works.  What people do on their machines is their
business, but I don't support or endorse or advertise multiple
rewrites of the kernel for the same reason that Stallman did not
accept this with Emacs.

'Stallman saw a problem in too much customization and de facto forking
and set certain conditions for usage. He later wrote

"EMACS was distributed on a basis of communal sharing, which means all
improvements must be given back to me to be incorporated and
distributed."  '

                                              https://en.wikipedia.org/wiki/Emacs#History

I'm not that strict; improvements that lie outside the kernel are fair
game.  It's one of the rare moments where I've agreed with something
Stallman has said.

Mark

[Greg Spurrier]

Ah. I just realized that we've been talking about different things.

When you wrote "why not just write shen.shen->kl?", I--not realizing that there was already an existing shen.shen->kl function--interpreted that as "why not just implement shen.shen->kl as opposed to implementing it as a system function?". But, now I know that you meant "why not just use the existing shen.shen->kl function?"

I wasn't suggesting that I wanted to refactor eval/eval-without-macros in my own private copy of Shen. Rather, I was suggesting a way that the functionality contained in those functions that essentially implements a version of shen->kl could be extracted into a stand alone function and that eval/eval-without-macros could be then implemented in terms of that function. My intention, though, was that would be done in the official sources.

It appears that shen.shen->kl can only be used to compile define forms. What I'm looking for is something that I can invoke on an arbitrary shen expression read by read-from-file or read-from-string and then get back a list of the corresponding KLambda forms. Is there already such functionality?

Thanks,

Greg

[Mark Tarver]

On Jun 13, 4:36 pm, Greg Spurrier <greg.spurr...@gmail.com> wrote:
> Ah. I just realized that we've been talking about different things.
>
> When you wrote "why not just write shen.shen->kl?", I--not realizing that
> there was already an existing shen.shen->kl function--interpreted that as
> "why not just implement shen.shen->kl as opposed to implementing it as a
> system function?". But, now I know that you meant "why not just use the
> existing shen.shen->kl function?"

I did wonder why it was such an issue for you. Good that the
misunderstanding is removed.

Mark

[Mark Tarver]

> I wasn't suggesting that I wanted to refactor eval/eval-without-macros in
> my own private copy of Shen. Rather, I was suggesting a way that the
> functionality contained in those functions that essentially implements a
> version of shen->kl could be extracted into a stand alone function and that
> eval/eval-without-macros could be then implemented in terms of that
> function. My intention, though, was that would be done in the official
> sources.

shen.elim-def is what you want. However I'm not certain that will
supplant eval-without-macros because what it produces
is an s-expr. You need something that actually evaluates this
expression to get the form recognised by the native platform.

Mark