Re: Booleans being accepted as symbols

[Mark Tarver]

Changing the code here can be done in three ways.

1. By changing the code for set so that values cannot be given to
booleans.
2. By inserting a check into the compiler.
3 By putting a check into the typechecker.

1. IMO is not a good idea because assignments should be lightning
fast. 2. or 3, is feasible and 3. would be my preference. So it would
not be a KLambda issue.

Regarding the assignment of 37 to true, the situation here is
different from a single namespace language. and these issues are
discussed here.

http://www.shenlanguage.org/Documentation/shendoc.htm#Notes on the
Implementation of Kl

The relevant passage is
QUOTE
"In a dual namespace model, the 'defun' and 'set' are regarded as
creating an association between the symbol and something else; it is a
mapping not an assertion of identity.

Generally Qi and Shen require a dual namespace for symbols. The
reason for this is to do with the Qi evaluation strategy for symbols
which is that symbols are self-evaluating. In Qi the symbol 'f'
evaluates to itself. If we want to get at the value associated with
'f', we type '(value f)'. Hence f is not thought of as shorthand for
a value, but is merely a symbol to which objects (definitions, global
assignments etc) can be attached."
UNQUOTE

Since set is not asserting identity but simply setting up an
association between its argument and something else which is retrieved
by 'value' the is no logical reason why one should restrict the domain
of arguments to 'value' to symbols. Generally precedent and
implementations do make this restriction even in dual namespace
languages, but it is not essential. It is essential that KLambda be
able to set the value of any symbol. What it does beyond that is a
matter of choice.

Mark

On Mar 31, 11:00 pm, Greg Spurrier <greg.spurr...@gmail.com> wrote:
> I mentioned in another thread that my KLambda test suite verifies that the
> primitives raise an error when invoked with unexpected types. I've come
> across several cases in Shen-CLisp where a value is expected to be a
> symbol, but a boolean is accepted, e.g.:
>
>     (0-) (set true 37)
>     37
>
>     (1-) (value true)
>     37
>
> Contrast this with a string argument which raises an exception as expected:
>
>     (2-) (set "true" 37)
>     SYSTEM::SET-SYMBOL-VALUE: "true" is not a symbol
>
> This affects 'set', 'value', 'defun' and 'lambda'. Can this be fixed in a
> future version of Shen CL?
>
> Thanks,
> Greg

[jo...@mentics.com]

> The relative cost of enforcing the types of the K Lambda primitives it going
> to vary from platform to platform. If the intention is to aproach the ideal
> of "write once, run anywhere", then each platform must bear the cost of
> faithfully conforming to the spec.

There is a balance that must be struck at times. I don't know if this
principle applies in this specific case, but consider the following:

If the standard function is sufficiently slower than a host specific
optimized one, at least some developers will choose to use the host
specific optimized one. In so doing, the goal of WORA is missed
anyway. So... in some cases, lack of performance can lead to lack of
portability because of developer choices that come as a result.

[Mark Tarver]

I've put the symbol check into the typechecker - it must have dropped
out some time back. Well done Greg.

What Greg is doing here is very important (I had missed the odd
behaviour with 'absvector'), but I think this point is very valid.  If
people are penalised for writing in Shen too heavily they will migrate
elsewhere.

The question is what WORA means.  Ramil's JS port will perform string
concatenation with '+'.  Vasil had a Scheme port which AFAIK used
hashing to mimic 'set' in theory allowing I suppose assignments to
many kinds of object.

I believe Ramil said in defence that the behaviour with '+' did not
matter because string concatenation with '+' would be blocked by the
typechecker.  If we allow this defence it means that WORA is aiming at
perfect portability wrt type secure code (call this 'weak
portability'). Strong portability requires portability wrt unchecked
or uncheckable code.

I think we should insist on weak portability and strive towards
strong. The KLambda standardisation is important though.

Mark

[Mark Tarver]

If the primitive functions of Shen do not permit writing reasonably
efficient code, then all Shen programs will become 'second-class
citizens' on every platform they are used. Nor will compiler
optimisation help since the primitives are themselves inefficient.
 People will resort to splicing in native code and then all
portability goes out of the window.

I think Joel is right here. I think losing perhaps 50x on incrementing
a variable is too high a price to pay, because, as I've said, the
logic of assignments in a Lisp-2 language does not in principle limit
assignments to symbols.  It is enough to insist 'set' work on symbols
and leave the rest open, placing the symbol constraint in the type
checker.

BTW in the interests of being systematic, could we work through
KLambda alphabetically?  Let's title each thread by the name of the
primitive.  Start with absvector.

Mark

On Apr 2, 2:23 pm, Greg Spurrier <greg.spurr...@gmail.com> wrote:
> Joel does raise a good point. However, in the case of a host-specific
> optimized variant of a function, the developer makes an explicit choice to
> use that variant and sacrifice portability for the sake of performance. My
> concern is for the developers that inadvertently discover host-specific
> behavior, believe it to be standard, and only find out much later that it
> is non-portable.
>
> My opinion is that we should strive for strong portability and use the
> presence of type information to enable the compiler to produce more
> efficient code when possible. Making untyped and uncheckable code second
> class citizens with respect to portability seems wrong to me.
>
> Greg

[Mark Tarver]

I got your drift. What you are proposing is that the primitive 'set'
be revised to be tighter even if it is slow, but the backend can be
optimised because 90% of the time we know at compile time that the
global is not a boolean etc. and hence those slow 'set's can be
optimised to be fast because we can compile them out to fast code.

I'd agree there is merit in this but

1. There are some dynamically generated 'sets' e.g. (set (hd X) (tl
X)) which cannot be optimised.
2. W.O. type checking, the info on the type is not there to make any
optimisation.
3. We end up with a primitive that is inherently so inefficient that
porters must avoid it and compile around it whenever they can.
4. If we are aiming at weak portability then we don't have to rise to
this challenge. If we want strong portability we have to deal with
other challenges - like Ramil's concatenating '+' sign. Actually it
would be good to hear him on this.

You put down some good arguments though. Let's put this one down as
open for now.

Mark

On Apr 4, 4:57 am, Greg Spurrier <greg.spurr...@gmail.com> wrote:
> I was not clear enough in my wording and it is possible that the point that
> I was trying to make was not clear. By 'compiler', I have been referring to
> the KLambda-to-host compiler, not the Shen-to-KLambda compiler. I will use
> the term KLHC below to avoid confusion.
>
> Please allow me one more attempt at arguing in favor of requiring the first
> arugment to 'set' to be a symbol and explaining how I think this can be
> implemented at an acceptable cost. If you still disagree, then I will have
> given it my best shot and will fully embrace your decision.
>
> In the KLambda sources for Shen 9.2, there are 66 uses of 'set'. 62 of
> these have symbol literals as their first argument. For those, the KLHC can
> emit the most efficient 'set' code possible because it knows that no
> run-time check is necessary. In the other 4 cases (one of which is a
> lexically-scoped variable reference; the others are function applications),
> the KLHC can emit a run-time check to validate that the first argument has
> evaluated to a symbol, followed by the actual set code.
>
> To put this in more concrete terms, I am suggesting that the CLisp
> implementation of KLambda transform (set X Y) to either:
>
>     `(SET ,X ,Y)
>
> or something along the lines of:
>
>     `(LET ((*C* ,X))
>           (IF (OR (EQ *C* 'true) (EQ *C* 'false))
>                  (ERROR "~A is not a symbol" *C*)
>                  (SET *C* ,Y)))))
>
> depending on whether X is a known symbol literal. The run-time check
> maximizes performance by taking advantage of the fact that SET already
> raises an error if its argument is not a Common Lisp symbol (see NOTE
> below). It just needs to reject the Common Lisp symbols 'true and 'false.
> An added check with SYMBOLP could of course be added to unify the error
> messages.
>
> We should, of course, question how often these run-time checks will be
> encountered in practice. I instrumented the ShenRuby compiler and run-time
> environment to gather this data. During the loading of the KL sources for
> Shen, 'set' is invoked 1,646 times. Zero run-time checks occur. During the
> loading and execution of the Shen test suite, 'set' is invoked 203,602
> times. Only 10 of these invocations required run-time checks. (Note, there
> is one test that does not run to completion because of stack overflow, so
> other's numbers may vary slightly if this experiment is repeated).
>
> Given these statistics, the additional overhead for a run-time check in the
> targeted cases that require them will have a negligible impact on the
> overall performance of the system. Therefore, I am of the opinion that the
> benefits for portability outweigh the performance impact.
>
> NOTE: the run-time check in my example code above is slightly more
> permissive than I would like. It will accept any Common Lisp symbol other
> than 'true and 'false. Given that 'intern' is allowed to produce
> non-boolean values that will return false for 'symbol?' there are other
> valid Common Lisp symbols that should be rejected. This could be fixed with
> a more expensive run-time check (it is perhaps this check that you are
> referring to as being 50X slower than SET), or by tightenening the spec for
> 'intern' so that it raises an error if given a string that would produce a
> value that satisfies neither 'boolean?' nor 'symbol?'. I prefer the latter
> because it also enhances portability, but IMHO, the former is also
> acceptable given the infrequency of it coming into play.
>
> Regards,
> Greg

[Ramil Farkhshatov]

> 4. If we are aiming at weak portability then we don't have to rise to
> this challenge. If we want strong portability we have to deal with
> other challenges - like Ramil's concatenating '+' sign. Actually it
> would be good to hear him on this.

I don't think that such quirks are worth dealing with. They do not
violate spec but only fall into `undefined behaviour` clause. Adding
checks like this in dynamic typed language means that arguments will be
checked twice: in kl-runtime and in target language runtime. It doesn't
increase code's reliability much and Shen have typechecking for that.