Macros in REPL vs compiled modules

[Unix One]

Apologies in advance for a potential newbie question. I'm having simple
fun with macros in LFE REPL:

lfe> (defmacro make-test-fun (string) `(defun test-fun() (io:format
,string)))
make-test-fun
lfe> (make-test-fun "123")
test-fun
lfe> (test-fun)
123ok
lfe> (make-test-fun "1234")
test-fun
lfe> (test-fun)
1234ok

Very cool!

But how do I do this inside a compiled module? It appears I can use
defmacro that defines a function and have it expanded into a function
during compile time. But (1) I cannot redefine functions as I did in
REPL, (2) how would that work in Erlang VM during runtime anyway?

If it can't be done, how close can I get?

Thank you.

[Robert Virding]

Hi,

Starting with modules. LFE modules are by necessity the same as Erlang modules as this is what the BEAM, the erlang VM, provides. Each module is basically a static collection of functions which are defined when the module is created and compiled (modules must be compiled). After this it is impossible to add or delete functions from a module without recompiling the *whole module*. This means we don't ave packages in the CL sense, or multiple namespaces either for that matter as there is only one namespace. You typically define the module in a file, a .lfe file, and compile it. You can define macros inside file and use them when defining functions inside the file/module. You can also import files containing macro definitions when you need to share them. There is a way to export macros so they can be called from other modules.

The REPL is slightly different. While the static handling of functions/macros in a module would work in the REPL it would be way too static to be useful. So *in the shell* you can dynamically create and recreate functions/macros and when you call a function/macro you get the latest version. Note that the REPL functions and macros are NOT contained in a module but are in a local database maintained by the REPL so there is no REPL module. This means that they cannot be called from normal modules. Why not have a REPL module? Well you can run multiple REPLs with different functions/macros which would make it very difficult to do something sensible.

I hope that helps explain some things,

Robert

[Unix One]

Hi Robert,

Thank you for your detailed answer, and even preemptively answering some
of my potential followup questions.

A couple of thoughts:

> After this it
> is impossible to add or delete functions from a module without
> recompiling the *whole module*.

Specifically in the case of redefining functions, does this mean that
BEAM does not have enough - i.e. the compiled module and the function's
new definition and whatever other data it already keeps about the module
- to hypothetically automatically recompile and reload the whole module
itself?

As for my last question, in the absence of such feature - how close can
I get? I can see lots of references for performing releases and hot code
loading, but none for self-updating code. If we ignore external
dependencies, then "all" we'd have to do would be:

- keep module source code
- expose function/macro for replacing/adding/removing functions
- replace function definition with the new one in module source
- compile new source
- load new module

most of which can be done by importing macro(s), right? Am I missing
something? Has anything similar been done already?

Thank you.