Defining functions within functions?

[Marcin Borkowski]

Hi all,

I have a long function with quite a few (even nested) lambdas inside.
I decided to refactor it so that it becomes more readable. Since I use
lexical scoping, simply changing the lambdas into defuns defined
elsewhere won't work.

In Scheme, one would probably use define inside another define. Is it
a good idea to use a (cl-)defun within a defun in Elisp to obtain
a closure in this situation? If not, what is a better way?

TIA,

--
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

[Eric Abrahamsen]

Marcin Borkowski <mb...@mbork.pl> writes:

> Hi all,
>
> I have a long function with quite a few (even nested) lambdas inside.
> I decided to refactor it so that it becomes more readable. Since I use
> lexical scoping, simply changing the lambdas into defuns defined
> elsewhere won't work.
>
> In Scheme, one would probably use define inside another define. Is it
> a good idea to use a (cl-)defun within a defun in Elisp to obtain
> a closure in this situation? If not, what is a better way?

Sounds like a job for cl-labels!

Eric

[Emanuel Berg]

Marcin Borkowski <mb...@mbork.pl> writes:

> In Scheme, one would probably use define
> inside another define. Is it a good idea to
> use a (cl-)defun within a defun in Elisp to
> obtain a closure in this situation? If not,
> what is a better way?

Create a new file and put all defuns on the
same level in that file.

Otherwise `cl-labels' but I'm hesitant if this
really makes the code more readable... (Not
compared to isolated defuns, for sure.)

--
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
- so far: 39 Blogomatic articles -

[Michael Heerdegen]

Marcin Borkowski <mb...@mbork.pl> writes:

> I have a long function with quite a few (even nested) lambdas inside.
> I decided to refactor it so that it becomes more readable. Since I use
> lexical scoping, simply changing the lambdas into defuns defined
> elsewhere won't work.
>

> In Scheme, one would probably use define inside another define. Is it
> a good idea to use a (cl-)defun within a defun in Elisp to obtain
> a closure in this situation? If not, what is a better way?

There is no simple answer to that question I think.

Sure, you could use `cl-labels', but that makes the code not much more
readable than `let' with lambdas.

Personally, I use `let' with lambdas most of the time for local function
definitions. When things get too complicated (seems you are at that
point), instead of using lexical closures as local functions, you can
use top-level defuns accepting additional arguments, or higher-level
functions (also as top-level defuns) that take the essential values as
arguments and return a closure.


Michael.

[Emanuel Berg]

Michael Heerdegen <michael_...@web.de>
writes:

> Personally, I use `let' with lambdas most of
> the time for local function definitions.
> When things get too complicated (seems you
> are at that point), instead of using lexical
> closures as local functions, you can use
> top-level defuns accepting additional
> arguments, or higher-level functions (also as
> top-level defuns) that take the essential
> values as arguments and return a closure.

Here is some terminology to wade thru - let's
see if I get it right:

"lexical closures as local functions" are the
same as `let' and lambdas with the lexical
scope enabled, which is the same as what is
normally thought of as local functions,
invisible and unavailable from anywhere else
but from within the function where they are
defined...

"top-level defuns" are normal defuns as they is
only one level in Elisp. (By the way, this is
the reason for the elaborate, prefixed naming
conventions, some-package-yada-yada-do-it, and
the like.)

What do you mean by "additional arguments" tho?
What I can see (?) the only thing replaced (not
added) is the let binding, by the
function name!

"higher-level functions" aren't at the level of
the long-lost High Ones, but on the same level
as the mere "top-level" Wolfrider defuns,
because "higher-level" refers to those function
accepting functions as arguments, as they are
not on any other level scope-wise than
the defuns.

What do you mean "return a closure" tho?
A closure is an association between a function
and its scope - but higher-level functions can
just as well return a normal value, e.g.

(defun eat-list (operator &rest args)
(apply operator args) )

(eat-list '+ 1 2 3 4) ; 10
(eat-list '* 1 2 3) ; 6
(eat-list '+) ; 0 <-- cool, btw
(eat-list '*) ; 1 <--

Perhaps if you provide an example, it'll be
easier grasp the "scope" of it :)

--
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic

- so far: 40 Blogomatic articles -

[Barry Margolin]

In article <864m9oj...@student.uu.se>,

Emanuel Berg <embe...@student.uu.se> wrote:

> What do you mean by "additional arguments" tho?
> What I can see (?) the only thing replaced (not
> added) is the let binding, by the
> function name!

If you have a local function, it can access variables in the calling
function directly. If you move it out to a top-level function, you have
to pass those variables as arguments.

(defun foo1 (x)
(flet ((local-bar (y)
(+ x y)))
(local-bar 3)))

(defun foo2 (x)
(global-bar 3 x))
(defun global-bar (x y)
(+ x y))

And if the function being called needs to reassign the variable, you
need to do that in the caller if you use a top-level function.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Emanuel Berg]

Barry Margolin <bar...@alum.mit.edu> writes:

> If you have a local function, it can access
> variables in the calling function directly.
> If you move it out to a top-level function,
> you have to pass those variables
> as arguments.

Right, I didn't think of that but actually it
is even more confusing.

It is more clear to call the function at top
level, and pass it all it needs to do its job.

> And if the function being called needs to
> reassign the variable, you need to do that in
> the caller if you use a top-level function.

"reassign the variable"?

[Barry Margolin]

In article <864m9ni...@student.uu.se>,

Emanuel Berg <embe...@student.uu.se> wrote:

> Barry Margolin <bar...@alum.mit.edu> writes:
>
> > If you have a local function, it can access
> > variables in the calling function directly.
> > If you move it out to a top-level function,
> > you have to pass those variables
> > as arguments.
>
> Right, I didn't think of that but actually it
> is even more confusing.
>
> It is more clear to call the function at top
> level, and pass it all it needs to do its job.
>
> > And if the function being called needs to
> > reassign the variable, you need to do that in
> > the caller if you use a top-level function.
>
> "reassign the variable"?

(defun foo-local (a)
(flet ((bar ()
(incf a)))
(bar))
a)

vs.

(defun foo-global (a)
(setq a (bar a))
a)
(defun bar (a)
(+1 a))

[Marcin Borkowski]

On 2016-05-23, at 07:34, Eric Abrahamsen <er...@ericabrahamsen.net> wrote:

> Marcin Borkowski <mb...@mbork.pl> writes:
>
>> Hi all,
>>

>> I have a long function with quite a few (even nested) lambdas inside.
>> I decided to refactor it so that it becomes more readable. Since I use
>> lexical scoping, simply changing the lambdas into defuns defined
>> elsewhere won't work.
>>
>> In Scheme, one would probably use define inside another define. Is it
>> a good idea to use a (cl-)defun within a defun in Elisp to obtain
>> a closure in this situation? If not, what is a better way?
>

> Sounds like a job for cl-labels!

Or cl-flet. But thanks anyway (and I learned about cl-flet when reading
about cl-labels!).

> Eric

Best,

[Marcin Borkowski]

On 2016-05-23, at 22:09, Michael Heerdegen <michael_...@web.de> wrote:

> Marcin Borkowski <mb...@mbork.pl> writes:
>
>> I have a long function with quite a few (even nested) lambdas inside.
>> I decided to refactor it so that it becomes more readable. Since I use
>> lexical scoping, simply changing the lambdas into defuns defined
>> elsewhere won't work.
>>
>> In Scheme, one would probably use define inside another define. Is it
>> a good idea to use a (cl-)defun within a defun in Elisp to obtain
>> a closure in this situation? If not, what is a better way?
>

> There is no simple answer to that question I think.
>
> Sure, you could use `cl-labels', but that makes the code not much more
> readable than `let' with lambdas.

True (at least to some extent - it just relocates the "unreadability"
elsewhere).

> Personally, I use `let' with lambdas most of the time for local function
> definitions. When things get too complicated (seems you are at that
> point), instead of using lexical closures as local functions, you can
> use top-level defuns accepting additional arguments, or higher-level
> functions (also as top-level defuns) that take the essential values as
> arguments and return a closure.

And I wanted to use the former variant, but it struck me as not very
elegant. As for the latter, I'm not sure I understand it exactly, but
I'll give it some thought.

For now, I decided to go with lambdas, but also to sprinkle the code
with comments. Old-fashioned, but should do the jon in my case.

> Michael.

Thanks,

[Emanuel Berg]

Barry Margolin <bar...@alum.mit.edu> writes:

>> "reassign the variable"?
>
> (defun foo-local (a)
> (flet ((bar ()
> (incf a)))
> (bar))
> a)
>
> vs.
>
> (defun foo-global (a)
> (setq a (bar a))
> a)
> (defun bar (a)
> (+1 a))

OK, so it is a situation with global variables?

Personally I never (?) use `setq' in defuns so
again this situation is hard to visualize in an
applied setting, but yes, I see
the reassignment.

[Emanuel Berg]

Marcin Borkowski <mb...@mbork.pl> writes:

> For now, I decided to go with lambdas, but
> also to sprinkle the code with comments.
> Old-fashioned, but should do the jon in
> my case.

... why is that old-fashioned?

It is much better to put everything in a new
file, and have all it all defuns on the
same level.

If you have comments, you are halfway there as
you can put the comments into docstrings!

Actually the `checkdoc-current-buffer'
I mentioned in another thread will tell you to
do that, turn the comment into a docstring, if
it suspects the comment serves the same purpose
as would a docstring.

Docstrings are better because they are
integrated in the help and interface
documentation system!

[Drew Adams]

> And I wanted to use the former variant, but it struck me as not very
> elegant. As for the latter, I'm not sure I understand it exactly, but
> I'll give it some thought.
>

> For now, I decided to go with lambdas, but also to sprinkle the code
> with comments. Old-fashioned, but should do the jon in my case.

A suggestion: Post a concrete example of what you need, and
see what concrete suggestions you get.

Typically, this stuff is not complicated. The first thing
to do, IMO, is to determine whether you really need/want to
do something special/complicated. Why do you think you want
a nested defun or other form of local function definition?

[Carlos Konstanski]

My 2 cents: as a common lisp propgrammer, I find two uses for
labels/flet:

- Recusrion. The wrapping code sets up the initial enviornment and then
calls the inner function where all the real work happens. In all other
languages you need two functions, but in lisp you can package the
entire thing into one form with an inner function.

- Arguments for free: If the flet is defined within the lexical scope
of some bindings, you can treat those bindings as "globals" and not
have to pass them into the function. I'm guessing this is what all the
let-over-lambda talk is about.

Both of these are valid uses of labels. Unless you have code
portability concerns, don't shy away from a package that gives emacs
lisp the same power as the other lisps.

Carlos

[Emanuel Berg]

Drew Adams <drew....@oracle.com> writes:

> A suggestion: Post a concrete example of what
> you need, and see what concrete suggestions
> you get.
>
> Typically, this stuff is not complicated.
> The first thing to do, IMO, is to determine
> whether you really need/want to do something
> special/complicated. Why do you think you
> want a nested defun or other form of local
> function definition?

100% correct!

People do it all the time. The describe the
problem in terms of what technology they think
will solve it. It makes it difficult for them
and everyone else.

But worst thing is even "professional"
employers do this. "We look for two Java
programmers with experience in Eclipse..." shut
up! Tell me your problem and I'll solve it
instantly by replacing your system with a truly
professional one!

*curses!!!*

PS. No disrespect to the OP who is into Lisp
intricacies for the sake of it like
many others here.

[Emanuel Berg]

Carlos Konstanski

<ckons...@pippiandcarlos.com> writes:

> My 2 cents: as a common lisp propgrammer,
> I find two uses for labels/flet:
>
> - Recusrion. The wrapping code sets up the
> initial enviornment and then calls the inner
> function where all the real work happens.
> In all other languages you need two functions,
> but in lisp you can package the entire thing
> into one form with an inner function.

Recursion is cool for schoolbook problems like
list and trees that are searched and shaped
back and forth. In most applications it is
better to use set functions (built-in
higher-order functions) to "do" the list
data...

> - Arguments for free: If the flet is defined
> within the lexical scope of some bindings,
> you can treat those bindings as "globals" and
> not have to pass them into the function.
> I'm guessing this is what all the
> let-over-lambda talk is about.

There is nothing wrong with passing things to
functions! On the contrary. It is much clearer
than having scope intermingling like
you describe.

[Marcin Borkowski]

On 2016-05-24, at 23:56, Drew Adams <drew....@oracle.com> wrote:

>> And I wanted to use the former variant, but it struck me as not very
>> elegant. As for the latter, I'm not sure I understand it exactly, but
>> I'll give it some thought.
>>
>> For now, I decided to go with lambdas, but also to sprinkle the code
>> with comments. Old-fashioned, but should do the jon in my case.
>

> A suggestion: Post a concrete example of what you need, and
> see what concrete suggestions you get.
>
> Typically, this stuff is not complicated. The first thing
> to do, IMO, is to determine whether you really need/want to
> do something special/complicated. Why do you think you want
> a nested defun or other form of local function definition?

The reason is simple: I have this long, almost 60-lines function, with
three levels of lambdas nested. (I'm using request.el, and I have to
make two requests sequentially, so I have a callback inside a callback.
I could probably use deferred or something, this is one of the ideas.)
It's just pretty complicated, and changing anything in this code is
a headache. Also, edebugging is simpler with separate instrumentable
defuns.

Best,

[Marcin Borkowski]

On 2016-05-25, at 00:10, Carlos Konstanski <ckons...@pippiandcarlos.com> wrote:

> Am Dienstag, den 24.05.2016, 14:56 -0700 schrieb Drew Adams:
>> >

>> > And I wanted to use the former variant, but it struck me as not
>> > very

>> > elegant.As for the latter, I'm not sure I understand it exactly,

>> > but
>> > I'll give it some thought.
>> >
>> > For now, I decided to go with lambdas, but also to sprinkle the
>> > code

>> > with comments.Old-fashioned, but should do the jon in my case.

>> A suggestion: Post a concrete example of what you need, and
>> see what concrete suggestions you get.
>>

>> Typically, this stuff is not complicated.The first thing

>> to do, IMO, is to determine whether you really need/want to

>> do something special/complicated.Why do you think you want

>> a nested defun or other form of local function definition?
>

> My 2 cents: as a common lisp propgrammer, I find two uses for
> labels/flet:
>
> - Recusrion. The wrapping code sets up the initial enviornment and then
> calls the inner function where all the real work happens. In all other
> languages you need two functions, but in lisp you can package the
> entire thing into one form with an inner function.

That I don't need.

> - Arguments for free: If the flet is defined within the lexical scope
> of some bindings, you can treat those bindings as "globals" and not
> have to pass them into the function. I'm guessing this is what all the
> let-over-lambda talk is about.

And those I do use.

> Both of these are valid uses of labels. Unless you have code
> portability concerns, don't shy away from a package that gives emacs
> lisp the same power as the other lisps.

If you mean (require 'cl-lib), I do it anyway. And if /I/ didn't,
request.el does.

> Carlos

[Stefan Monnier]

> The reason is simple: I have this long, almost 60-lines function, with
> three levels of lambdas nested. (I'm using request.el, and I have to
> make two requests sequentially, so I have a callback inside a callback.
> I could probably use deferred or something, this is one of the ideas.)
> It's just pretty complicated, and changing anything in this code is
> a headache. Also, edebugging is simpler with separate instrumentable
> defuns.

These kinds of callback codes are often described as "CPS-style code",
and in some cases a couple of macros can go a long way to hiding this
machinery and give the illusion of "plain old sequential code", reducing
the nesting madness.


Stefan

[Nicolas Petton]

Stefan Monnier <mon...@iro.umontreal.ca> writes:

> These kinds of callback codes are often described as "CPS-style code",
> and in some cases a couple of macros can go a long way to hiding this
> machinery and give the illusion of "plain old sequential code", reducing
> the nesting madness.

I guess generator.el can also help.

Nico

[to...@tuxteam.de]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, May 25, 2016 at 04:36:16AM -0400, Stefan Monnier wrote:

> These kinds of callback codes are often described as "CPS-style code",
> and in some cases a couple of macros can go a long way to hiding this
> machinery and give the illusion of "plain old sequential code", reducing
> the nesting madness.

Nevertheless: debugging is still torture, right?

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAldFa1oACgkQBcgs9XrR2kYszACfeH3tgY1yvvauZf2E2NqAA+7w
qosAnAmNTNsj9kyc4aEmeyZ8rqC5bjOr
=RGpp
-----END PGP SIGNATURE-----

[Drew Adams]

> > A suggestion: Post a concrete example of what you need, and

^^^^^^^^

> > see what concrete suggestions you get.
> >

> > Typically, this stuff is not complicated. The first thing

> > to do, IMO, is to determine whether you really need/want to

> > do something special/complicated. Why do you think you want

> > a nested defun or other form of local function definition?
>

> The reason is simple: I have this long, almost 60-lines function, with
> three levels of lambdas nested. (I'm using request.el, and I have to
> make two requests sequentially, so I have a callback inside a callback.
> I could probably use deferred or something, this is one of the ideas.)
> It's just pretty complicated, and changing anything in this code is
> a headache. Also, edebugging is simpler with separate instrumentable
> defuns.

I repeat the suggestion above, in case it helps: _concrete_ example.

And preferably try to show the problem using a simplified version of
your code (maybe 60 lines of code is not needed to point out your
problem(s)/question). Show just what you need to show to indicate
the problem/question.

My guess is that you might get more help in this way than with only
an abstract characterization of what you think you need, or an
abstract description of the problem or of your code. Maybe
continuation-passing is really the best approach in your case; but
maybe it is not (needed)...

You can do lots of things with Emacs Lisp that the language is not
necessarily best designed for. Emacs Lisp is not Scheme or Haskell,
even if you can (sort of) fake doing some of what you do all the
time with such languages. Over time, Elisp has gotten a little
closer (e.g. closures/lexical binding in general, better compiling),
but it is still quite a different beast.

And though it is hard (for me) to consider your question at only
an abstract level, that does not mean that it won't help _you_
to look at it abstractly.

Finally (in case it helps), when I see an Emacs-Lisp question
"Defining functions within functions?", I wonder first whether it
might be an X-Y question/problem: asking about a tried solution
when the more useful question might be about the problem you are
trying to solve. It is not super common in Emacs Lisp to define
functions within other function definitions. My first (admittedly
reflex) question is whether that is really what you need/want.
http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem

Just a suggestion.

[Marcin Borkowski]

On 2016-05-25, at 16:18, Drew Adams <drew....@oracle.com> wrote:

>> > A suggestion: Post a concrete example of what you need, and
> ^^^^^^^^
>> > see what concrete suggestions you get.
>> >
>> > Typically, this stuff is not complicated. The first thing
>> > to do, IMO, is to determine whether you really need/want to
>> > do something special/complicated. Why do you think you want
>> > a nested defun or other form of local function definition?
>>
>> The reason is simple: I have this long, almost 60-lines function, with
>> three levels of lambdas nested. (I'm using request.el, and I have to
>> make two requests sequentially, so I have a callback inside a callback.
>> I could probably use deferred or something, this is one of the ideas.)
>> It's just pretty complicated, and changing anything in this code is
>> a headache. Also, edebugging is simpler with separate instrumentable
>> defuns.
>
> I repeat the suggestion above, in case it helps: _concrete_ example.

I agree, though it will take me some time to distill the problem into
a kind of minimal example. I'll try to get back here when I do it.

Thanks,

[Marcin Borkowski]

On 2016-05-25, at 11:07, to...@tuxteam.de wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Wed, May 25, 2016 at 04:36:16AM -0400, Stefan Monnier wrote:
>
>> These kinds of callback codes are often described as "CPS-style code",
>> and in some cases a couple of macros can go a long way to hiding this
>> machinery and give the illusion of "plain old sequential code", reducing
>> the nesting madness.
>
> Nevertheless: debugging is still torture, right?

Especially with request.el... In case of (some types of) errors in
callbacks, requests just fails completely silently, with no error
message whatsoever...

Best,

[Marcin Borkowski]

On 2016-05-25, at 10:43, Nicolas Petton <nic...@petton.fr> wrote:

> Stefan Monnier <mon...@iro.umontreal.ca> writes:
>
>> These kinds of callback codes are often described as "CPS-style code",
>> and in some cases a couple of macros can go a long way to hiding this
>> machinery and give the illusion of "plain old sequential code", reducing
>> the nesting madness.
>

> I guess generator.el can also help.

Thanks, I looked into it and I don't see how, but maybe I can't see something.

> Nico

[Emanuel Berg]

Marcin Borkowski <mb...@mbork.pl> writes:

>> I repeat the suggestion above, in case it
>> helps: _concrete_ example.
>
> I agree, though it will take me some time to
> distill the problem into a kind of minimal
> example. I'll try to get back here when
> I do it.

No more LaTeX for you! The example doesn't have
to be "minimal" - you can post the entire chunk
of code if you want.

But even so, start by turning all lambdas or
what have you into defuns. That shouldn't take
long with finger habits, killing and yanking.

PS. By the way, speaking of "concrete" stuff -
in Hitler's Germany, the Nazi had a method
of identifying Communists. They did it like
this. They didn't ask any questions on
politics or the international situation.
Instead they just taked about common day
things. If they noticed that the person got
restless, and especially if [s]he used the
word "concrete", they were dead certain
[s]he was a Communist :)

PPS. Oh, no! I brought the Nazis into the
argument. Doesn't that mean
I automatically lost it, or is that rule
no longer in effect?