Remove currying from the standard?

[deech]

Hi all,
What are people's thoughts on removing the currying requirement from the standard?
(1) it can always be replaced with an explicit lambda so to me it's just syntactic convenience and more importantly,
(2) it has some pretty surprising behavior when interacting with macros, for instance:

(3-) (+ 100 100)
200
(4-) (+ 100)
[closure [[A0 | 100] t] [A1] [apply [function shen/+] [list A0 A1]]]
(5-) (defmacro addmacro [+ N] -> [+ N 10])
addmacro
(6-) (+ 100)
110

Thanks!
-deech

[Mark Tarver]

Actually Deech;  I don't think that's very smart.  Go away and think about it a bit.  If you want to go in that direction - rewrite the types and the proofs in TBoS.

Mark

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.
To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

[deech]

I did think about it before posting and yes, I get that it's being used all over the place and would break existing code and documentation. If I'm missing some fundamental point about how currying is more than sugar for a lambda please let me know. Otherwise I'm arguing that there's some potential catastrophic pitfalls with macro interaction and it would be a good idea to migrate away from it in the long term.

On Saturday, October 7, 2017 at 12:38:20 PM UTC-5, Mark Tarver wrote:

Actually Deech;  I don't think that's very smart.  Go away and think about it a bit.  If you want to go in that direction - rewrite the types and the proofs in TBoS.

Mark

On Sat, Oct 7, 2017 at 6:20 PM, deech <aditya...@gmail.com> wrote:

Hi all,
What are people's thoughts on removing the currying requirement from the standard?
(1) it can always be replaced with an explicit lambda so to me it's just syntactic convenience and more importantly,
(2) it has some pretty surprising behavior when interacting with macros, for instance:

(3-) (+ 100 100)
200
(4-) (+ 100)
[closure [[A0 | 100] t] [A1] [apply [function shen/+] [list A0 A1]]]
(5-) (defmacro addmacro [+ N] -> [+ N 10])
addmacro
(6-) (+ 100)
110

Thanks!
-deech

--
You received this message because you are subscribed to the Google Groups "Shen" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.

[Mark Tarver]

You can use macros to change the laws of arithmetic if you want; to make 1 + 1 = 3 - apparently.  This doesn't really undermine currying or arithmetic or any other convention.  Whats it shows is that macros in Shen have near unlimited capability.  

Macros were built into Shen during the bootstrapping development process to allow a very limited language, Kl, to express an apparently much richer one, Shen.

If your example has a point, it is in favour of a device for taming the power of macros for the average applications programmer.  Such a device would sit on the Shen reader and circumscribe the power of macros and it would be a macro!    A start is implicit in the discussion of the use and misuse of macros in TBoS.  I'd start there.

However you'd probably lose something in doing so.  My remarks about ljnear separability in my 2017 video apply - see 16:42 and after on the video on the Shen home

page.  

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

[deech]

I understand the power of Shen macros. It's a very bold design choice and I like it! I'm not looking to neuter them in any way.

All I'm saying is that there's this really neat and innovative thing about Shen (macros) that's currently clashing with this really simple thing that's just syntax sugar (currying) that most people won't really miss so why not plan to get rid of it?

Currying also complicates the porting process especially in the case of curried anonymous functions where the arity is not recorded. The user takes a performance hit because I have to keep checking if the thing is still a closure before applying the next argument. It also complicates essential static analysis tooling such as code completion.

And all this for very little gain in terms of expressivity or design. The feature just doesn't earn its keep.

[Neal Alexander]

From the perspective of generating machine code: removing currying from the standard would make the low level calling convention more efficient, or at the very least, a lot more simple.

Look at this page on Haskell's function calling conventions: https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/FunctionCalls

The same applies to Shen, except Shen has more dynamically typed code so the run-time type checks and branching would be much more common.

Of course you aren't obligated to implement function calls the same way as they do. If you do real currying, and all functions are actually unary then you don't have to test and branch so much. The downside is that you generate code with too many indirect jumps to functions that do nothing, killing instruction caches and pipelines in the process. It may be less of an issue for a bytecode interpreter though. 

If currying is just used as syntax sugar in 90% of cases, it may not be worth supporting real currying at the runtime level. Do any of the ports implement it without some type of performance penalty?

However, even if you remove currying you still have to deal with functions of unknown arity.

[Mark Tarver]

Macros are nothing to do with this.  Deech I've just explained to you why in two posts.  Currying is not 'syntactic sugar' - it is integral to the semantics of the type theory and the proofs.  We shouldn't be having this conversation.    Take a chill pill and go off read the book.

Mark 

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

[Bruno Deferrari]

On Sun, Oct 8, 2017 at 11:08 AM, deech <aditya...@gmail.com> wrote:

I understand the power of Shen macros. It's a very bold design choice and I like it! I'm not looking to neuter them in any way.

All I'm saying is that there's this really neat and innovative thing about Shen (macros) that's currently clashing with this really simple thing that's just syntax sugar (currying) that most people won't really miss so why not plan to get rid of it?

Currying also complicates the porting process especially in the case of curried anonymous functions where the arity is not recorded. The user takes a performance hit because I have to keep checking if the thing is still a closure before applying the next argument. It also complicates essential static analysis tooling such as code completion.

And all this for very little gain in terms of expressivity or design. The feature just doesn't earn its keep.

deech, can you show me a snippet of code that you find troublesome to compile efficiently?

The few cases where I can't fully resolve statically are because of dynamic typing, not currying.

 

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[deech]

No, I think I'll "go away" for now and take that prescribed "chill pill". I'm done here.

BD

[Antti Ylikoski]

As for some benefits of currying:

https://en.wikipedia.org/wiki/Currying

AJY

[Bruno Deferrari]

On Sun, Oct 8, 2017 at 10:08 AM, Neal Alexander <nha...@gmail.com> wrote:

From the perspective of generating machine code: removing currying from the standard would make the low level calling convention more efficient, or at the very least, a lot more simple.

Look at this page on Haskell's function calling conventions: https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/FunctionCalls

The same applies to Shen, except Shen has more dynamically typed code so the run-time type checks and branching would be much more common.

Of course you aren't obligated to implement function calls the same way as they do. If you do real currying, and all functions are actually unary then you don't have to test and branch so much. The downside is that you generate code with too many indirect jumps to functions that do nothing, killing instruction caches and pipelines in the process. It may be less of an issue for a bytecode interpreter though. 

If currying is just used as syntax sugar in 90% of cases, it may not be worth supporting real currying at the runtime level. Do any of the ports implement it without some type of performance penalty?

However, even if you remove currying you still have to deal with functions of unknown arity.

No support is required at runtime, at least not in the current versions of the Shen Kernel (earlier versions were more complicated, but those were implementation accidents, not something that the KLambda Spec required). I would even argue that currying support does make things easier, not harder. There is only one situation in KLambda where the function call cannot be resolved at compile time, it looks like this**:

(Op Arg1 Arg2 Arg3)

which can be translated into:

(((Op Arg1) Arg2) Arg3)

leaving you with a single case that needs to be handled at runtime:

(Op Arg)

All this case requires at runtime, is a check to see if `Op` is indeed a function before applying it, something you get "for free" (in terms of implementation, not performance) if you are implementing Shen on top of a dynamic language platform.

But you have to make this check at runtime anyway because of dynamic typing support in Shen, even if currying wasn't in the picture, you would not be able to avoid this check.

This is all I do in shen-scheme, I never do an arity check at runtime (no need to, since every lambda is a 1-place function), I only use arity information at compile-time to do transformations like the one described above. Half of the code of the compiler is for handling function applications, but it isn't that long, and KLambda being a small language as it is, there isn't much else other than function applications:

https://github.com/tizoc/shen-scheme/blob/chez/src/compiler.shen#L151-L254

I don't take advantage of type information (I have played with it, but never integrated it into the compiler), but it could be used to optimise some situations (if not all) in programs that have been verified as type secure.

** `Op` can be an expression instead of a variable, but it can be reduced into this case:

(<exp> ...) -> (let Op <exp> (Op ...)

 

 

On Saturday, October 7, 2017 at 7:20:40 PM UTC+2, deech wrote:

Hi all,
What are people's thoughts on removing the currying requirement from the standard?
(1) it can always be replaced with an explicit lambda so to me it's just syntactic convenience and more importantly,
(2) it has some pretty surprising behavior when interacting with macros, for instance:

(3-) (+ 100 100)
200
(4-) (+ 100)
[closure [[A0 | 100] t] [A1] [apply [function shen/+] [list A0 A1]]]
(5-) (defmacro addmacro [+ N] -> [+ N 10])
addmacro
(6-) (+ 100)
110

Thanks!
-deech

--

You received this message because you are subscribed to the Google Groups "Shen" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[Bruno Deferrari]

On Sun, Oct 8, 2017 at 1:10 PM, Bruno Deferrari <uti...@gmail.com> wrote:

On Sun, Oct 8, 2017 at 10:08 AM, Neal Alexander <nha...@gmail.com> wrote:

From the perspective of generating machine code: removing currying from the standard would make the low level calling convention more efficient, or at the very least, a lot more simple.

Look at this page on Haskell's function calling conventions: https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/FunctionCalls

The same applies to Shen, except Shen has more dynamically typed code so the run-time type checks and branching would be much more common.

Of course you aren't obligated to implement function calls the same way as they do. If you do real currying, and all functions are actually unary then you don't have to test and branch so much. The downside is that you generate code with too many indirect jumps to functions that do nothing, killing instruction caches and pipelines in the process. It may be less of an issue for a bytecode interpreter though. 

If currying is just used as syntax sugar in 90% of cases, it may not be worth supporting real currying at the runtime level. Do any of the ports implement it without some type of performance penalty?

However, even if you remove currying you still have to deal with functions of unknown arity.

No support is required at runtime, at least not in the current versions of the Shen Kernel (earlier versions were more complicated, but those were implementation accidents, not something that the KLambda Spec required). I would even argue that currying support does make things easier, not harder. There is only one situation in KLambda where the function call cannot be resolved at compile time, it looks like this**:

(Op Arg1 Arg2 Arg3)

To clarify, here `Op` is a variable (which means that at runtime, the value can be anything), all I have written in my previous email doesn't apply to cases where `Op` is just a symbol, in those cases you have all the information you need at compile-time.

--

BD

[Neal Alexander]

In the case of (((Op Arg1) Arg2) Arg3):

The above is the simplest solution, but a runtime arity check plus a saturated call to 'Op' is much faster than having to create and destroy 3 activation records, plus jump to 3 different code locations. However, because we need to support currying it means that we need to check if Arg3 needs to be applied to the result of (Op Arg1 Arg2) if Op is arity 2, instead of just throwing an error.

The situation of Op not being known comes up in every (untyped) higher order function, which doesn't seem that rare.

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.

To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[Bruno Deferrari]

On Sun, Oct 8, 2017 at 3:59 PM, Neal Alexander <nha...@gmail.com> wrote:

In the case of (((Op Arg1) Arg2) Arg3):

The above is the simplest solution, but a runtime arity check plus a saturated call to 'Op' is much faster than having to create and destroy 3 activation records, plus jump to 3 different code locations. However, because we need to support currying it means that we need to check if Arg3 needs to be applied to the result of (Op Arg1 Arg2) if Op is arity 2, instead of just throwing an error.

The situation of Op not being known comes up in every (untyped) higher order function, which doesn't seem that rare.

Yes, I was talking about runtime simplicity. If performance is an issue, there are two things that I think will help you much more than getting rid of currying:

- Use static typing information so that you know at compile-time that the operator is indeed a function (along with it's arity), so that you can avoid such checks, and produce better code to make the dynamic function call.

- Use inlining. Not a solution for big functions, but helps with a lot of common cases, like `map`, `filter`, and similar small higher-order functions. You generate a bit more code, but now the function call is statically known, letting you use the fastest option for calling the function.

Both those, alone and in combination let you improve performance in a lot of other cases too, like when doing arithmetic, something that has a lot of overhead when no type information is available.

These are generic optimisations that can be done at the KLambda level and that can be implemented as a library once, and reused by all ports (same as the transformation shen-scheme does that I mentioned in the previous email).

Ramil's KLVM already provides some of these generic transforms: https://github.com/gravicappa/klvm

 

To unsubscribe from this group and stop receiving emails from it, send an email to qilang+unsubscribe@googlegroups.com.

To post to this group, send email to qil...@googlegroups.com.
Visit this group at https://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[Samuel Falvo II]

On Saturday, October 7, 2017 at 12:44:55 PM UTC-7, deech wrote:

I did think about it before posting and yes, I get that it's being used all over the place and would break existing code and documentation. If I'm missing some fundamental point about how currying is more than sugar for a lambda please let me know. Otherwise I'm arguing that there's some potential catastrophic pitfalls with macro interaction and it would be a good idea to migrate away from it in the long term.

Perhaps it would help to enumerate the perceived problems you refer to.  What are these negative interactions you speak of?  You give one example, but it's hard for me to see why this is a problem, given such a small and isolated example.  For now, I can only interpret it as an artifact of how macros work; it's not actually a problem until it gives me grief somehow.  Can you list how this causes you or other programmers problems?  Once that's known, maybe alternative idioms can be proposed which aren't so disruptive, or folks can reconsider given the wider understanding.

Thanks.