Match Expression

[Robert Koeninger]

I've long felt a feature that's been lacking in Shen is the ability to do pattern matching in places other than function parameters. It's annoying to have that functionality at the start of a define and then have to revert to writing hd/tl calls later on in the same function.

Here's a prototype for a match expression, re-using the parser code for functions. It uses F# keywords but takes multiple argument expressions like a Shen function. Contrived usage:

(match List N with

  _        0 -> 0

  [X | Xs] N -> (+ (* X N) (recur Xs N))

  []       N -> (error "empty list"))

(no patterns matched fails like a partial function - prompts for tracking)

I noticed there's a macro to do pattern matching in lambdas in the shen-libs repo, and this would be in the same spirit. (does this still work with newer versions of the kernel? "shen-<define>").

Should these things just be in a library or part of the kernel? It feels arbitrary to have only global function parameter pattern matching built-in, but not these other 2 forms.

[Mark Tarver]

Pattern matching is one of those features which exists in multiple forms in many languages from segment matching to sorted pattern matching to equational matching to higher-order unification etc.  Too many to put in the standard.  Generally the best way to do this is to use Shen macros to put them in a library.  TBoS shows how to do this (p 111, 117).  

I'd cite http://www.marktarver.com/Philosophy/index.htm   maxim 4. and Perlis's maxim about program design

https://quotefancy.com/quote/1363580/Alan-Perlis-In-the-long-run-every-program-becomes-rococo-and-then-rubble .  Keep the kernel simple.  

A thread on this question from the past.  I agree with Vasil's commnets.

https://groups.google.com/forum/?hl=en#!searchin/qilang/Vasil$20pattern/qilang/Lu5_m6e1gCs/IG9iFZ3S7ZoJ

Mark

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.
To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

[Robert Koeninger]

"Keep the kernel simple" sounds like a status quo bias. You could also keep it simple by removing pattern matching entirely. It could just be a library, right? But since it's already in there, it stays.

In my mind, it's not just about keeping it simple, but well-rounded. Once you've added one feature (function parameter matching), it doesn't make sense to leave out a closely-related feature (match expression). In fact, if you could only pick one, it would make more sense to include a match expression to the exclusion of parameter matching, as it can be placed anywhere, including in nested expressions and lambdas. And it wouldn't conflate function definitions and pattern matching.

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.

[Bruno Deferrari]

On Mon, Jan 8, 2018 at 5:44 PM, Robert Koeninger <rkoen...@korewireless.com> wrote:

I've long felt a feature that's been lacking in Shen is the ability to do pattern matching in places other than function parameters. It's annoying to have that functionality at the start of a define and then have to revert to writing hd/tl calls later on in the same function.

Here's a prototype for a match expression, re-using the parser code for functions. It uses F# keywords but takes multiple argument expressions like a Shen function. Contrived usage:

(match List N with

  _        0 -> 0

  [X | Xs] N -> (+ (* X N) (recur Xs N))

  []       N -> (error "empty list"))

(no patterns matched fails like a partial function - prompts for tracking)

I noticed there's a macro to do pattern matching in lambdas in the shen-libs repo, and this would be in the same spirit. (does this still work with newer versions of the kernel? "shen-<define>").

No, it has two issues, shen-<define> is now called shen.<define>, and it passes a symbol instead of a function (that works in SBCL and maybe other ports too, but it is not portable).

But it is easy to fix: https://github.com/vasil-sd/shen-libs/pull/21/files

 

Should these things just be in a library or part of the kernel? It feels arbitrary to have only global function parameter pattern matching built-in, but not these other 2 forms.

--

You received this message because you are subscribed to the Google Groups "Shen" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[Bruno Deferrari]

On Mon, Jan 8, 2018 at 9:12 PM, Bruno Deferrari <uti...@gmail.com> wrote:

On Mon, Jan 8, 2018 at 5:44 PM, Robert Koeninger <rkoen...@korewireless.com> wrote:

I've long felt a feature that's been lacking in Shen is the ability to do pattern matching in places other than function parameters. It's annoying to have that functionality at the start of a define and then have to revert to writing hd/tl calls later on in the same function.

Here's a prototype for a match expression, re-using the parser code for functions. It uses F# keywords but takes multiple argument expressions like a Shen function. Contrived usage:

(match List N with

  _        0 -> 0

  [X | Xs] N -> (+ (* X N) (recur Xs N))

  []       N -> (error "empty list"))

(no patterns matched fails like a partial function - prompts for tracking)

I noticed there's a macro to do pattern matching in lambdas in the shen-libs repo, and this would be in the same spirit. (does this still work with newer versions of the kernel? "shen-<define>").

No, it has two issues, shen-<define> is now called shen.<define>, and it passes a symbol instead of a function (that works in SBCL and maybe other ports too, but it is not portable).

But it is easy to fix: https://github.com/vasil-sd/shen-libs/pull/21/files

Note that one limitation of that version is that references to variables in the outer scope will not work.

 

 

Should these things just be in a library or part of the kernel? It feels arbitrary to have only global function parameter pattern matching built-in, but not these other 2 forms.

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.
To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

--

BD

[Mark Tarver]

Pattern matching is intimately linked to the sequent calculus so it really is not optional.  As said;  programmers vary about what they want in a language and what not.  For example, Vasil did not like the very features you want and thought it made for ugly code (I actually agree with his views).  The Shen macro system allows you to add these things very easily.  It was invented precisely to allow people to do their own stuff.  So if you feel you want these features, code them as macros and make the code available and people who like it will use it.

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

[Tatsuya Tsuda]

How about putting your pattern matching library in a port dependent package and include it in your port as a kind of port dependent system function if you think that is essential?

Since there are many porters and it is really hard for people to agree on new features, if I were you I would start from that. 

If many ports include the same feature and it shows its importance, it may become a standard feature at some point.

I know that you care about the standard, but I have seen people getting unmotivated and leave the community because of their passion not accepted, and I don't want to see that again.

As a person who likes Clojure and Common Lisp rather than a small language like Scheme R5RS, I really care about practicality and usefulness of the language and I prefer to include essential features in a port rather than an external library.

Making the standard simple and small is great, but porters have different requirements and I think porters should be allowed to include different features in your own port.

Tatsuya

2018年1月9日火曜日 8時59分57秒 UTC+9 Robert Koeninger:

[Mark Tarver]

Generally every programmer has some kind of itch that they want to scratch because every programmer has his or her ideas about what they find important or what they like.  And they do not agree.   Willi for instance loves FOR statements.   You cannot put these things in the standard because otherwise you'll end up with PL/1.   You can put them in a port, but then only the people who use that port will be able to use them.  Put them in an external library and everyone in every port can use them.   This is what macros were invented for.    C for example, is not a large language, but has an extensive standard library.  

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

[Mark Tarver]

I will tell you one additional very good reason why you should not smuggle experimental stuff into the standard kernel.   It can really screw up portability.   For example, if you introduce some new function f into the kernel and systemf it, you can make problems you never envisaged.   This has already happened with Willi who has had to rewrite part of his program because the new OS Shen introduced a function not in TBoS and defined in his program.  His program would not load.

Putting these additions into a standard library and out of the kernel is a big help to application writers.  Really that is where the effort should go; as I've been saying for a long time.

Mark

[Mark Tarver]

This is not a criticism of the code itself btw.  It is a question of where you introduce it.   Innovation is good as long as you know where it is.

Mark

[Tatsuya Tsuda]

We can include a new feature in a port and also make a copy as an external library. But this does not solve the portability issue you mention.

I know that portability is an important reason for creating Shen, but it is still unclear to me how important it is.

I think there is always a trade off between portability and convenience.

For instance, in Shen-JVM there is a function called sj.println that I frequently use for debug purpose. Should I get rid of this and not include in the port? 

The reason I included is that I frequently use this function and I am too lazy to load an external library when I open the REPL.

If there is an official rule to not include new features to a port then we can get rid of the problem that Willi faced.

I think that if portability is the most important thing then we should not allow new features to be included in a port and it should be written in an official document that the porters can see to avoid further confusion.

Or if there are more relaxed rules or such, I would like to know that too.

Tatsuya

2018年1月9日火曜日 20時53分30秒 UTC+9 Mark Tarver:

[Robert Koeninger]

Tatsuya,

Yeah, I also think we should avoid depending on port-specific features. Not only would it result in non-portable code, but non-portable libraries, and then libraries based on those libraries will be non-portable and Shen becomes 10 different languages.

[Douglas Miles]

On Tue, Jan 9, 2018 at 2:05 PM, Robert Koeninger <rkoen...@korewireless.com> wrote:

Tatsuya,

Yeah, I also think we should avoid depending on port-specific features. Not only would it result in non-portable code, but non-portable libraries, and then libraries based on those libraries will be non-portable and Shen becomes 10 different languages.

Makes sense, ​can Shen support original feature as an "installable extension"?  ( That is can be added to the library in a way that will work from port to port? )

 

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

[Robert Koeninger]

Douglas,

Depends on what you mean. If the feature only depends on what's in the kernel, then sure, and it can be completely portable, and many things you'd think of as fundamental language features can be introduced this way.

If you mean libraries that depend on port-specific features that can work for multiple ports (a file system library that does a different set of FFI calls per port), there doesn't appear to be a standard for that yet. There's an issue on the GitHub project about adding feature conditionals: macros that include code based on port, operating system or other environmental factors. I was also thinking there could be a standard of putting port-specific code in files with suffixes like ".cl.shen", ".js.shen", etc. which are only loaded for that port. This allows port-specific code to be isolated from everything that is completely portable.

[Mark Tarver]

Well its very important because that was the raison d'etre of Shen over Qi.

If there is an official rule to not include new features to a port then we can get rid of the problem that Willi faced.
I think that if portability is the most important thing then we should not allow new features to be included in a port and it should be written in an official document that the porters can see to avoid further confusion.

I'd agree with this.  Bog standard programs like Willi's should not fail because of creative additions to the kernel.   We (Willi and I) are going through all the ports systematically and we'll highlight what we find.    Willi has some of the most comprehensive and complex Shen programs around.

I think that adding new system functions to the kernel, however well-intentioned, should not be done.   These things belong in a standard library.    We have a good one in SP and I'm working  on the financial side so as to be able to release it.   

I also think we could merge the SP kernel with the OS kernel.   This is mutually beneficial; the two are very close.

For instance, in Shen-JVM there is a function called sj.println that I frequently use for debug purpose. Should I get rid of this and not include in the port? 
The reason I included is that I frequently use this function and I am too lazy to load an external library when I open the REPL.

Every port will contain port-specific functions involved in building the backend and coding the primitives.   If these are hidden in a package,  and the logical place is the shen package, 

I see no harm in that.   Making the functions external to a package is what causes problems.

We're clearing the decks for the arrival of the OS Ring and we'll see in the coming months the fruition of a lot of good work.

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

[Bruno Deferrari]

On Wed, Jan 10, 2018 at 10:44 AM, Mark Tarver <dr.mt...@gmail.com> wrote:

Well its very important because that was the raison d'etre of Shen over Qi.

If there is an official rule to not include new features to a port then we can get rid of the problem that Willi faced.
I think that if portability is the most important thing then we should not allow new features to be included in a port and it should be written in an official document that the porters can see to avoid further confusion.

I'd agree with this.  Bog standard programs like Willi's should not fail because of creative additions to the kernel.   We (Willi and I) are going through all the ports systematically and we'll highlight what we find.    Willi has some of the most comprehensive and complex Shen programs around.

I think that adding new system functions to the kernel, however well-intentioned, should not be done.   These things belong in a standard library.    We have a good one in SP and I'm working  on the financial side so as to be able to release it.   

I added some functions to the kernel in the last releases, some of which I want to remove from the kernel in the next release, and others which I plan to keep. I'm interested in knowing what other's thoughts are on this:

Keep:

- dict functions

  these just expose a variation of something that was already inside the Shen kernel, and are used internally. Other than the datatype being useful, the main motivation for adding these was allowing ports to override them with their native implementation of hash tables, which helps with performance considerably.

- value/or, get/or, etc ("*/or" variants of already existing functions)

  these are just variants of already existing accessor functions, what accept a continuation to be used as the default value when the access fails. The motivation for these was performance. In some (most?) ports exception handling is not that cheap, and get a big performance boost from having this option.

- read-char-code

- read-file-as-charlist

  these two were added to fix issues with Shen/SBCL when compiled using recent versions of SBCL (>= 1.1.2) on Windows, which has multibyte I/O. The default implementation just calls read-byte, but for ports where it doesn't work, it can be overriden (here is Shen/SBCL's and Shen/CCL's implemention for example: https://github.com/Shen-Language/shen-cl/blob/6517e26f4232e46cfb233d9ac35856f73e92c07b/src/overwrite.lsp#L122-L127 )

- sterror

  it is like stoutput, but returns the STDERR stream. Not used much on Windows, but it is almost impossible to write useful command-line programs in other platforms without this. Could be removed and just made a convention, but it being part of the kernel means that the Kernel can take advantage of it when printing errors, to avoid conflicting with the program's output.

- for-each

  this is just like map, but doesn't keep a list of results around. Replaced internal uses of `map` where the result was dropped and helped with performance a bit.

Remove:

- fold-left

- fold-right

- filter

  all useful, but no reason to be in the kernel, they are not used internally (and if the where, they can be kept internal and not exposed).

- exit

- command-line

  useful for portability, but no reason to be in the kernel either, all is needed is for ports to follow a convention (e.g. if a function to access command-line arguments is added to a port, it should be named command-line so that it can be used portably).

--

BD

[Mark Tarver]

IMO you need an OS standard lib - much of this stuff would find a place in it.  Then people can access it w.o. having Willi's problems.   Any internal changes to the kernel for the sake of performance is fine.   The SBCL stuff probably needs its own package sbcl.  

Basically the global namespace in the kernel should be kept clear for system functions that exist in the standard.  Platform-specific extensions belong in platform-specific packages and extensions that are coded in Shen and generally useful for everybody belong in stlib.   sterr may need thought.   

You've done a first-class job BTW in creating Chez/Shen.    This is just detail.

Mark

[Bruno Deferrari]

On Wed, Jan 10, 2018 at 11:53 AM, Mark Tarver <dr.mt...@gmail.com> wrote:

IMO you need an OS standard lib - much of this stuff would find a place in it.  Then people can access it w.o. having Willi's problems.   Any internal changes to the kernel for the sake of performance is fine.   The SBCL stuff probably needs its own package sbcl.  

Basically the global namespace in the kernel should be kept clear for system functions that exist in the standard.  Platform-specific extensions belong in platform-specific packages and extensions that are coded in Shen and generally useful for everybody belong in stlib.   sterr may need thought.   

My proposal for the next release, that keeps most of the benefits and without losing much (or any) of the gained performance:

* Anything from the "remove" list goes away.

* Only sterror stays global. I can think of different solutions for this one, but nothing that I consider as good and simple as keeping it.

* for-each is made internal.

* Dict functions are either:

A) made internal.

B) moved to their own namespace -- either "dict", or something "private" like "$$dict".

Whatever it is, names will be documented so that platforms can overwrite with native versions. If it is a "private" namespace, a "dict" library can be provided that maps the names to something more accessible.

Remember that even if dicts become a library (from the user point of view) they are still used internally by the Kernel (for performance), which means they will always be loaded (a private namespace would "hide" them to avoid possible clashes).

* read-char-code and read-file-as-charlist can just be internal and documented. Any program that intends to work with multibyte streams and character encodings will need something much more advanced anyway.

* "*/or" functions gone. This will cause performance to degrade at first, but it is something that ports can fix by improving their Kl->Native compilers (it is a little bit more work, but very doable).

--

BD

[Mark Tarver]

Sounds good,   I'll drop you a line.

bw

Mark