[Caml-list] Examples where let rec is undesirable

[Diego Olivier Fernandez Pons]

List,

I was wondering if there was any reason not to make "let rec" the default /
sole option, meaning cases where you clearly don't want a "let rec" instead
of "let" (only in functions, not cyclic data).

Diego Olivier

--
Caml-list mailing list. Subscription management and archives:
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[Alexandre Pilkiewicz]

Hi Diego.

I think one of the best reason to *not* have let rec as the default is
shadowing.

Say you have a function that expect a string as an argument

let foo s =
....
....

and you realize that it's the job of the callee and not the caller to
check that the string is escaped. You can just change it to:

let foo s =
let s = escape s in
....
....

And before you say "let's just do this for function and not for other
values", let's remember that nothing separate a function from a value
syntactically (it's a functional language, a function *is* a value),
only typing can. And you can't type your program before you know its
structure, and let vs let rec is part of the structure.

Then on top of that we can add that it makes compilation faster,
optimization easier, and code clearer (and, of course, backward
compatibility :))!

Cheers

Alexandre


On Mon, Jan 2, 2012 at 5:37 PM, Diego Olivier Fernandez Pons
<dofp....@gmail.com> wrote:
> � � List,

[Lukasz Stafiniak]

On Mon, Jan 2, 2012 at 11:37 PM, Diego Olivier Fernandez Pons
<dofp....@gmail.com> wrote:
>     List,
>
> I was wondering if there was any reason not to make "let rec" the default /
> sole option, meaning cases where you clearly don't want a "let rec" instead
> of "let" (only in functions, not cyclic data).
>
>          Diego Olivier

The default "no-rec" allows for name recycling -- using the same name
for an incrementally transformed value, i.e. to bind the intermediate
results. Name recycling minimizes the cognitive burden: there are less
names to remember in a scope, and differences in names are justified
by differences in purpose of the values. Are there reasons to consider
name recycling a bad style?

Happy New Year,
Łukasz

[Martin Jambon]

On 01/02/2012 04:05 PM, Lukasz Stafiniak wrote:
> On Mon, Jan 2, 2012 at 11:37 PM, Diego Olivier Fernandez Pons
> <dofp....@gmail.com> wrote:
>> List,
>>
>> I was wondering if there was any reason not to make "let rec" the default /
>> sole option, meaning cases where you clearly don't want a "let rec" instead
>> of "let" (only in functions, not cyclic data).
>>
>> Diego Olivier
>
> The default "no-rec" allows for name recycling -- using the same name
> for an incrementally transformed value, i.e. to bind the intermediate
> results. Name recycling minimizes the cognitive burden: there are less
> names to remember in a scope, and differences in names are justified
> by differences in purpose of the values. Are there reasons to consider
> name recycling a bad style?

What I am wondering is why type definitions are recursive by default. It
is mostly troubling for beginners.

The following is an occasional inconvenience of having only recursive
type definitions:

module A =
struct
type t = ...
let compare = ...

module B =
struct
type t = t (* uh oh *)
let compare = compare
end
end


Martin

[Gabriel Scherer]

> What I am wondering is why type definitions are recursive by default. It
> is mostly troubling for beginners.

Indeed, it is a defect of the language not to have non-recursive type
definitions. My usual reference on this topic is Yaron's complaint:
https://ocaml.janestreet.com/?q=node/25

The problem is not so much that they are "recursive by default" (I
believe this would be justified by the relative frequencies of
recursive type definitions and type-level name recycling) but that
they *must* be recursive. It would be better to have some kind of
"type-nonrec" construct for non-recursive type declarations.

This is however a case were it is difficult to fix the issue in a
fully convenient way: you don't want to add a new keyword, you
certainly don't want to change the behavior of existing "type"
declarations, and all solutions remaining are likely to be convoluted
and feel like a ad-hoc patch.
Given there is the reasonable (if ugly) workaround of breaking the
cycle through an auxiliary type renaming, there are few incentives to
change the state of the art. I believe this is something you could and
should easily fix when designing a new language (when introducing
*any* binding construct, think about both a recursive (when it makes
sense) and a non-recursive version), but that may have to stay with us
for OCaml.

[Yaron Minsky]

For what it's worth, my feeling is that even if there weren't technical
reasons to require the "rec'' marking for recursive functions, I would want
it as a language feature. Recursive definitions are harder to understand
than non recursive ones, and it's helpful to have the static guarantee that
only definitions so marked will be allowed to refer to themselves. This
makes it hard to mistakenly refer to yourself, which I think is a real
source of error.

y
On Jan 2, 2012 5:38 PM, "Diego Olivier Fernandez Pons" <dofp....@gmail.com>
wrote:

[Richard W.M. Jones]

On Tue, Jan 03, 2012 at 01:05:39AM +0100, Lukasz Stafiniak wrote:
> On Mon, Jan 2, 2012 at 11:37 PM, Diego Olivier Fernandez Pons
> <dofp....@gmail.com> wrote:
> >     List,
> >
> > I was wondering if there was any reason not to make "let rec" the default /
> > sole option, meaning cases where you clearly don't want a "let rec" instead
> > of "let" (only in functions, not cyclic data).
> >
> >          Diego Olivier
>
> The default "no-rec" allows for name recycling -- using the same name
> for an incrementally transformed value, i.e. to bind the intermediate
> results. Name recycling minimizes the cognitive burden: there are less
> names to remember in a scope, and differences in names are justified
> by differences in purpose of the values. Are there reasons to consider
> name recycling a bad style?

I had an argument about this with a noted open source developer
recently. He was saying that C's approach -- not permitting variable
names to be reused within a single function -- was somehow
advantageous. From my point of view, having used both languages
extensively, OCaml's way is *far* better.

So yes, 'let' and 'let rec', long may they be different.

Rich.

--
Richard Jones
Red Hat

[ivan chollet]

Allowing variable shadowing is aesthetically more satisfying and more
expressive, but opens the door to bugs that can be harder to track by
static analysis.
I would be interested to hear the pro-arguments for variable shadowing,
besides the slight gain in expressiveness.

[Gabriel Scherer]

My argument is the following: when you make a *local* edition to a
piece of code, shadowing allow you to pick variable name without
having to know about what's already bound in context. If you can't
shadow, you have to keep in mind what is in the context, which
increases the cognitive burden (during code edition) and also the
maintenance cost in some situations, such as "moving this
(self-contained) piece of code to this place (possibly with a
different context)".

I have also found language that disallow shadowing to be a pain in
practice, and you can find concrete examples of this in Erlang and, to
a lesser extent, Haskell. In Erlang, you will often find patterns that
look (once translated to ML) like this
let x1 = foo x in
let x2 = bar x1 in
let x3 = blah x2 in
foobar x3
(In Haskell those don't appear as is because you will use function
composition or a State monad; but you can still find "x1", or "x
prime", as a common and awkward idiom)

This style is painful to maintain: if you suddently wish to add an
intermediate step, you have important non-local change to make, which
increase the opportunity for bugs, or at least source inconsistencies.


The flipside of my meta-argument above (disallowing shadowing increase
context-dependency a lot) is that some people argue that the problem
is not with forbidding shadowing, but with having a non-trivial
outside context. Bindings, they argue, should not be nested, and you
should never have more than two, three layers of scope in your code.
Making non-trivial environments harder is nearly an explicit goal of
their language design choice, and they are therefore more than happy
to forbid shadowing. For example, that the position of the
Coffeescript designer (a syntactic sugar layer of Javascript which is
getting popular for using indentation over braces, but has made the
imho. awful choice of making it nearly impossible to define a local
variable). It is a rational choice, but I still disagree strongly; I
find all arguments of the shape "you will never have more than N of
these things, or you're guilty of doing something wrong" dubious.

[Richard W.M. Jones]

On Thu, Jan 05, 2012 at 08:27:25PM +0000, ivan chollet wrote:
> Allowing variable shadowing is aesthetically more satisfying and more
> expressive, but opens the door to bugs that can be harder to track by
> static analysis.

You'll have to explain a bit more why this is.

Also, why is variable shadowing bad within a function (in C), but just
fine across different functions? Surely if it's bad at all, every
local variable in the whole program should have a unique name?

[Richard W.M. Jones]

On Thu, Jan 05, 2012 at 09:46:32PM +0100, Gabriel Scherer wrote:
> Bindings, they argue, should not be nested, and you
> should never have more than two, three layers of scope in your code.

I often think the problem here is not with the code, nor the
programmers, but with the tools. Editors should make it easier to
fold scopes away and annotate the folded-away scopes with comments,
when you don't want to directly look at the code.

[ivan chollet]

Sorry Richard I should have elaborated a bit more.
I guess there are a couple of examples in the literature, but one of them
comes to my mind, consider the following code snippet:

let fd = Unix.open "myfile1" ... in
let fd = Unix.open "myfile2" ... in
.. (some code)
Unix.close fd

This causes a file descriptor leak that is hard to detect statically in
general. This happened to me, i tend to avoid variable shadowing since then.
As a rule of thumb, I think it's better to give different conceptual
objects different variable names, which also improves self-documentation.
Within nested scopes, all objects declared with a let-binding are usually
distinct conceptually.
Across independent scopes, of course code fragments are independent and
variable naming schemes are independent, however within nested scopes, my
opinion is that variable shadowing is bad practice, for the reason
exemplified above.
More generally, one of the reason why developers like languages like Caml
is that features like strong typing, ADT, pattern matching prevents them to
be bitten by their own code, while still preserving a high level of
expressiveness. Because there is no free lunch, this also increase the
cognitive burden on the developer at write-time, and is the price to pay to
write code with less bugs.

Hope this clarifies my point of view a little.

[Cedric Cellier]

> I often think the problem here is not with the code, nor the
> programmers, but with the tools.  Editors should make it easier to
> fold scopes away and annotate the folded-away scopes with comments,
> when you don't want to directly look at the code.

Most editors do this, at least when the syntax of the language makes it simple to delimit the scopes (ie for most languages but ML).

Not only folding but the whole editing of source code can be made more efficient when the editor understand the scoping rules, as paredit shown for lisp.

[David Allsopp]

ivan chollet wrote:
> Sorry Richard I should have elaborated a bit more.
> I guess there are a couple of examples in the literature, but one
> of them comes to my mind, consider the following code snippet:
>
> let fd = Unix.open "myfile1" ... in
> let fd = Unix.open "myfile2" ... in

> ... (some code)
> Unix.close fd

I got stung by this kind of thing a few years ago (no resource leaking, just referring to a variable which had been shadowed and wasn't the string/list I thought it was - I'm capable of being very dim). I posted about it in 2008 - http://caml.inria.fr/pub/ml-archives/caml-list/2008/08/41f896b99ecf9a84b3f2e977bbc4e232.fr.html

My own determination was that variable shadowing is fine as long as the type changes - because in most cases I found that errors were then caught by the type checker and the number of cases where I had to make variable names differ where I hadn't before was low enough.

I've quickly chucked the scripts at www.metastack.com/ocaml/checkshadow.tar.gz (the camlp4 filter is Gabriel's: http://bluestorm.info/camlp4/dev/pf_shadow/list.php). The only problem is that the camlp4 filter doesn't handle [if] expressions correctly, but it works by saying (for foo.ml):

ocamlfind ocamlopt -c -annot -package pf_shadow -syntax standard foo.ml && checkShadow foo

[pf_shadow.cmo is built with ocamlfind ocamlc -syntax camlp4o -package camlp4.lib,camlp4.quotations -c pf_shadow.ml and checkShadow with ocamlfind ocamlopt -c -package unix checkShadow.ml]

and emits warnings if it detects variable shadowing of the same type. I hasten to add, this was all done to deal with a personal deficiency - IMHO the language itself should not enforce this behaviour. I too find the complete ban on variable shadowing in other languages intensely irritating (especially when it's even between separate scopes within the same function)


David

[Daniel Bünzli]

> Sorry Richard I should have elaborated a bit more.
> I guess there are a couple of examples in the literature, but one of them
> comes to my mind, consider the following code snippet:
>
> let fd = Unix.open "myfile1" ... in
> let fd = Unix.open "myfile2" ... in

> ... (some code)

> Unix.close fd
>
> This causes a file descriptor leak that is hard to detect statically in general.

If you mean detect statically by a _program_, then this makes
absolutely no difference. A static analyzer could alpha rename the
term so that each name is unique or work with De Bruijn indexes
without changing the complexity of the problem.

If you mean detect statically by a _programmer_ then yes, variable
shadowing can be a good tool to write obfuscated code.

> As a rule of thumb, I think it's better to give different conceptual objects
> different variable names, which also improves self-documentation.

Agreed. (I'm sure also got caught at least once by something like your
first example).

> Within nested scopes, all objects declared with a let-binding are usually distinct
> conceptually.

Not necessarily. I think what Łukasz mentions about incrementally
transformed values and cognitive burden is very true and in these
cases I wouldn't consider name recycling a bad style.

One example is working with a purely functional datastructure like
Set.t where you need to perform a few applications before getting to
the only value you are interested in. The alternative would be to
simply not name the intermediate results, it may be possible but
sometimes writing the sequence of application explicitly by a serie of
lets makes the presentation and the code clearer [1].

Best,

Daniel

[1]
Somewhat related is point-free style programming of which haskellers
are very fond of. A little bit of it is nice, but I think it sometimes
makes the cognitive burden too high. There's a balance between
conciseness and readability. Somehow programmers are obsessed by the
former at the detriment of the latter, but a good balance is needed.
This example (from this [2] wikipedia page) speaks for itself :

mf criteria operator list = filter criteria (map operator list)

vs

mf = (. map) . (.) . filter

I'm sure we all agree the former is easier to grasp.

[2]
http://en.wikipedia.org/wiki/Point-free_programming

[Damien Doligez]

On 2012-01-05, at 21:46, Gabriel Scherer wrote:

> My argument is the following: when you make a *local* edition to a
> piece of code, shadowing allow you to pick variable name without
> having to know about what's already bound in context.

I agree with Gabriel. Consider the following expression:

(let x = 21 in x + x)

Assuming I haven't been so crazy as to redefine (+), is this
a valid expression whose value is 42?

First possible answer: yes
Second possible answer: it depends on the context.

I'd rather go with the first choice, and that means allowing
shadowing. Consider a camlp4 extension that produces such
expressions. Does it have to look at the context (including
the "open" statements, functor applications, first-class
modules, etc.) before it can insert this expression into your
code?

-- Damien