LFE startup and initialisation files

[Robert Virding]

The is an follow on to the discussion about extensions, if any, for LFE file and script files. In that discussion I also took up which startup files we should have, their names and where they should lie. It is this discussion I have broken out into a separate thread.

One suggestion we are working with is that we have a ~/.lfe *directory* where could have LFE files/libraries/etc which LFE would automatically find by default. This is useful. Some of these files could be init files. It has also been discussed that we could have multiple init files, both local to one LFE app and "global" to all LFEs run by a user. I think that this a good idea. We also have the user_default.erl module in which you can define your own shell functions. There is also the ~/.erlang file which erlang reads when it starts.

I think there should be 2 levels of initialisation for LFE: one level is for the LFE system which is always run; and one for each LFE shell. To support this I suggest we mirror emacs and have a ~/.lfe file for the LFE system with commands when LFE are started, and have a ~/.lfe.d directory where we can keep libraries and other initialisation files. The ~/.lfe file is similar to the ~/.erlang file and the ~/.emacs file and would contain expressions which are evaluated at LFE startup. There should also be startup files for the LFE shell which are run for every shell called 'lferc' or maybe 'lfeshrc'. There could a "global" one in the ~/.lfe.d directory and a local one in your current directory/application. Both would be loaded but the local one would shadow the global one. Maybe there should be a settable search path for the lferc files. The lferc files would contain general LFE shell commands not just LFE expressions so they would be more similar to .basrc files than to .erlang files.

I have always had the ideal that you can multiple shells which do different things and need different initialisation. Unfortunately parts of the erlang system don't support this, for example there is only one cwd, and there is not much to do about this. The user_default module has the same problem. Have local LFE shell initialisation files would at least allow you to have different per shell commands.

Maybe this is just over engineering and adds to many layers so it becomes difficult to keep track of what is going on?

This is just me thinking out aloud so I can judge my own ideas and get some feed-back. Please comment,

Robert

[Mason Staugler]

Are you proposing the following?

~/.lfe -- system initialization

~/.lfe.d/lfeshrc -- global shell initialization

./lfeshrc -- local shell initialization

user_default

The user_default support is good because it means the LFE shell inherits what I've got configured for my Erlang shell. I like having both lfeshrc's initialized because I can have both my idiosyncratic helper functions and project specific ones (for debugging, building, etc).

What is the ~/.lfe for? The only thing I can think of is adding to the code path, but .erlang is already serviceable for that.

(Since the files are shell specific I prefer lfeshrc over lferc.)

[Duncan McGreggor]

On Sat, Feb 14, 2015 at 11:57 AM, Robert Virding <rvir...@gmail.com> wrote:

The is an follow on to the discussion about extensions, if any, for LFE file and script files. In that discussion I also took up which startup files we should have, their names and where they should lie. It is this discussion I have broken out into a separate thread.

One suggestion we are working with is that we have a ~/.lfe *directory* where could have LFE files/libraries/etc which LFE would automatically find by default. This is useful. Some of these files could be init files. It has also been discussed that we could have multiple init files, both local to one LFE app and "global" to all LFEs run by a user. I think that this a good idea. We also have the user_default.erl module in which you can define your own shell functions. There is also the ~/.erlang file which erlang reads when it starts.

I think there should be 2 levels of initialisation for LFE: one level is for the LFE system which is always run; and one for each LFE shell. To support this I suggest we mirror emacs and have a ~/.lfe file for the LFE system

What about ~/.lfe/system (file)

 

with commands when LFE are started, and have a ~/.lfe.d directory

Couldn't that just use the ~/.lfe directory?

It might be worth cataloging what files/directories we'll want to be using ... for starters:

 * ~/.lfe

 * ~/.lfe/libs - global, user-level libraries for projects, shells, etc.; used by lfetool)

 * ~/.lfe/lfe.config - defaults for LFE configs, rebar overrides, rebar and relx configs, etc.

 * lfe.config - same as the previous, but per-project

 * ~/.lfe?rc - a run control config file for LFE shells (we could even do ~/.lfe/rc or ~/.lfe/shrc ...)

Regardless, I like your idea of splitting system and shell.

 

where we can keep libraries and other initialisation files. The ~/.lfe file is similar to the ~/.erlang file and the ~/.emacs file and would contain expressions which are evaluated at LFE startup. There should also be startup files for the LFE shell which are run for every shell called 'lferc' or maybe 'lfeshrc'. There could a "global" one in the ~/.lfe.d directory and a local one in your current directory/application. Both would be loaded but the local one would shadow the global one. Maybe there should be a settable search path for the lferc files. The lferc files would contain general LFE shell commands not just LFE expressions so they would be more similar to .basrc files than to .erlang files.

I have always had the ideal that you can multiple shells which do different things and need different initialisation.

I *really* like this idea. I'd like to have a custom shell for scientific computing needs, another for quick-access to LFE, another for prototyping projects, etc.

 

Unfortunately parts of the erlang system don't support this, for example there is only one cwd, and there is not much to do about this. The user_default module has the same problem. Have local LFE shell initialisation files would at least allow you to have different per shell commands.

Maybe this is just over engineering and adds to many layers so it becomes difficult to keep track of what is going on?

I think this is an appropriate amount of engineering :-) I live in the REPL -- I'd like to have it a comfortable, flexible, and powerful home :-) That takes good engineering!

d

 

This is just me thinking out aloud so I can judge my own ideas and get some feed-back. Please comment,

Robert

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.
To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at http://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

[Duncan McGreggor]

Quick amendment  below ...

On Sat, Feb 14, 2015 at 3:35 PM, Duncan McGreggor <dun...@cogitat.io> wrote:

On Sat, Feb 14, 2015 at 11:57 AM, Robert Virding <rvir...@gmail.com> wrote:

The is an follow on to the discussion about extensions, if any, for LFE file and script files. In that discussion I also took up which startup files we should have, their names and where they should lie. It is this discussion I have broken out into a separate thread.

One suggestion we are working with is that we have a ~/.lfe *directory* where could have LFE files/libraries/etc which LFE would automatically find by default. This is useful. Some of these files could be init files. It has also been discussed that we could have multiple init files, both local to one LFE app and "global" to all LFEs run by a user. I think that this a good idea. We also have the user_default.erl module in which you can define your own shell functions. There is also the ~/.erlang file which erlang reads when it starts.

I think there should be 2 levels of initialisation for LFE: one level is for the LFE system which is always run; and one for each LFE shell. To support this I suggest we mirror emacs and have a ~/.lfe file for the LFE system

What about ~/.lfe/system (file)

 

with commands when LFE are started, and have a ~/.lfe.d directory

Couldn't that just use the ~/.lfe directory?

It might be worth cataloging what files/directories we'll want to be using ... for starters:

 * ~/.lfe

 * ~/.lfe/libs - global, user-level libraries for projects, shells, etc.; used by lfetool)

 * ~/.lfe/lfe.config - defaults for LFE configs, rebar overrides, rebar and relx configs, etc.

 * lfe.config - same as the previous, but per-project

 * ~/.lfe?rc - a run control config file for LFE shells (we could even do ~/.lfe/rc or ~/.lfe/shrc ...)

I should have said that this was just an example ... I'd be just as happy with ~/.lfe.d/ being the place where we put everything. In other words, I'm advocating for consolidation of configuration/global/system files and directories in a primary location, as opposed to many files and directories that are siblings of each other in ~/ ... unless everyone hates that idea, in which case I'll follow the crowd ;-)

d

[Mário Guimarães]

Hello,

here my two cents:

./.lfe - an LFE configuration directory root

./.lfe/deps - dependency modules in source form or as precompiled beams

./.lfe/system.lfe (file) - general configuration of the LFE system

./.lfe/shell.lfe (file) - general configuration of the LFE shell

The *.lfe files would be LFE script files that would be run before anything when loading the LFE system or shell, respectively. In fact, it should be possible to run them from the terminal to launch an LFE daemon (system.lfe) or shell (shell.lfe) -- this duality daemon/shell would be similar, for example, to that of emacs daemon and emacsclient (note: this is just an workflow example, as I don't want for LFE to be tight by any means to any editor!)

The above directories would exist for the installation of LFE, for the user home, and for every project directory. When launching LFE via the "lfe" command, this would search the current directory, the user home, or the global installation directory for an .lfe root dir and run whichever of these files are there (note: if an .lfe root dir is found but one of these files is missing there, then the respective file in the user home or the installation LFE directory, whichever is found first by this order, would be run).

Also, if I wanted to override these files when launching LFE, it should be possible to do something like this:

$ lfe -system mysys.lfe -shell myshell.lfe

If I wanted to start some shell, it should be possible to do it simply by calling

$ myshell.lfe

or

$ lfe -shell myshell.lfe

and the contents of this file would be

#! ... lfe -shell

(... code starts here ...)

The code in all these files would be normal LFE code that would use a configuration API for the system and shell.

Finally, the "lfetool" would build on this to improve the LFE experience (e.g., to run builds, update dependencies, etc, much like similar tools for other languages, like maven and ant, but every configuration in this tool would be just more LFE code, i.e., LFE code everywhere). For example, the lfetool would create an ./lfe project dir like the above, by copying all configuration files from the user home or installation this, thus making the project directory self-contained (thus avoiding any "magical" configuration loaded from outside the project directory).

This is what I would like to have, as it would make configuring and using LFE simple, easy, versatile, consistent, and understandable.

I just want to help shape LFE as a wounderful, pleasant and serious tool for large scale development. I hope this little contribution helps.

Thanks,

Mário

[Mário Guimarães]

Hi

two more things:

1) There should be only two command line tools:

- the "lfe" command for launching system daemons, shells, and run scripts. This tool should enable access to every possible LFE configuration

- and the "lfetool" for supporting workflows, builds, and other project stuff, on top of "lfe" and its project directories. Note that it must be possible for developers to live without this tool, but this tool would add sweet sugar to the LFE experience.

2) There should be only one extension for LFE code files -- .lfe -- whether these are used as scripts or not. The "lfe" command line should be able to distinguish in which context and how code should be run.

In fact, it should be possible to run LFE code as a module or script without changes to keep things simple.

I think this would simplify LFE use, but you know better than I what can be done. I think that this is stuff that must be thought and done before LFE 1.0

Thanks

Mário 

[Mário Guimarães]

Hi,
I was expecting some comments to my last two messages ... did I miss something?
-mg

[Duncan McGreggor]

I don't think you missed anything. Your thoughts sound very similar to those that I've had and some other folks' too. I think there's general agreement on these matters and now it's a matter of trying ideas out.

From what I gather, Robert's got a branch where he's working on some of this stuff.

A couple of random notes:

 * I'm not sure how possible it would be to run a module as a script, or treat a script as a module -- these are handled quite differently right now, and may hinge on Erlang internals.

 * Your thoughts on the responsibilities of LFE vs. lfetool are *very* close to what Robert and I have been discussing (possibly identical)

I did read your emails when they came in, but I didn't feel I had anything else to add -- you had stated things well. When Robert has something ready to test, we can try it out and provide feedback on our experiences, and he can then better help us adapt our workflows to what is possible with LFE.

Thanks!

d

[Robert Virding]

Hi Mário,

No, you haven't missed anything I have just been a bit busy the last few days. Work rears it ugly head, well not that ugly actually. :-)

Just to expand a little on what Duncan said and some comments of my own:

- As Duncan said modules and scripts are very different beasts even though both contain LFE. A module can only contain function definitions and no other expressions or commands at all. An LFE script can contain LFE shell commands, expressions, function and macro definitions, etc, which are evaluated in the same way as if they had been entered in shell. Also by adding a '#! /usr/bin/env lfe' line at the beginning you can make a shell script an executable; this is what lfec does. This is why I was discussing having different extensions. So my suggestion is that LFE module files have a '.lfe' extension while an LFE script either has none or perhaps a '.lfesh' when you want to be explicit. That modules are like this is hard wired into erlang.

- I would prefer to call the library directory in ~/.lfe (or ~/.lfe.d) for lib instead of deps. This as it is a library directory rather than build dependencies and it follows the name convention of erlang.

- While lfe could determine whether a start file is a module definition, and call main/1, or a shell script I much prefer to say that the start files for the LFE shell *are* LFE script files. I don't see the point of allowing both. Even if you do have a module it won't be a general one anyway.

- I was thinking of the ~/.lfe (call it ~/.lferc perhaps, like .bashrc) file as being a general configuration file and the lfeshrc files as shell config files.

- While a general ~/.lfe file is not really necessary as we could just use the ~/.erlang file I think it would be nice to split the 2 worlds. Same with user_default module.

- One thing I am not decided on is if we have a sequence directories where lfe searches for an lfeshrc file when starting a shell does it stop at the first one it finds? Or does it load them all but those before in the sequence shadow the later ones?

- When you start by doing '$ lfe -shell myscript.lfeshrc' do you mean that the shell starts, loads in myscript and then gives you a prompt? And when you said start a shell by just doing '$ myscript' do you mean the same thing? That you get a shell prompt in both cases?

- You can today do '$ lfe myscript' or just '$ myscript' if the first line is '#! /usr/bin/env lfe' and the script file is loaded and executed but then lfe terminates. Just like running a sh/bash script file.

- Would these override the default ones, or complement them?

- Same questions for the -system options?

- I wonder if we need a -system option as it will do the same as the -shell anyway?

- Perhaps we could have an LFE_LIBS environment variable to add a load path for modules? In the same way that ERL_LIBS does.

- What problem are we trying to solve with the startup scripts? Are we making it too complex and should we just let people write shell scripts for the more complex cases?

- As you probably notice I like things to be explicit. :-)

Again I am thinking aloud here as well as commenting your suggestions? I will come with another suggestion in a few days, but please comment.

Robert

P.S. Lfescript, which is based on erlang's escript, basically has a module definition with a function main/1 which is called to do the work. I think the lfe command with a script file is a better option.

On Tuesday, 17 February 2015 00:37:34 UTC+1, Duncan McGreggor wrote:

I don't think you missed anything. Your thoughts sound very similar to those that I've had and some other folks' too. I think there's general agreement on these matters and now it's a matter of trying ideas out.

From what I gather, Robert's got a branch where he's working on some of this stuff.

A couple of random notes:

 * I'm not sure how possible it would be to run a module as a script, or treat a script as a module -- these are handled quite differently right now, and may hinge on Erlang internals.

 * Your thoughts on the responsibilities of LFE vs. lfetool are *very* close to what Robert and I have been discussing (possibly identical)

I did read your emails when they came in, but I didn't feel I had anything else to add -- you had stated things well. When Robert has something ready to test, we can try it out and provide feedback on our experiences, and he can then better help us adapt our workflows to what is possible with LFE.

Thanks!

d

On Mon, Feb 16, 2015 at 5:28 PM, Mário Guimarães <mario.luis.guimaraes@gmail.com> wrote:

Hi,
I was expecting some comments to my last two messages ... did I miss something?
-mg

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.
To post to this group, send email to lisp-flavoured-erlang@googlegroups.com.

[Mário Guimarães]

fine :-)

Regarding the module vs script dichotomy, it simply shouldn't exist.

For example, a script could be a module without an explicit module definition, and whose module name is that of the script, and it would not export any function, but when passed to the "lfe" command line this will search for a function named main and execute it, and if the main function does not exist nothing happens and "lfe" ends doing nothing or warns "main not found"

Now for a module to be executed in script mode the "lfe" command just needs to search for a main function and execute it, and this would not even need to be exported!

In fact, the "lfe" command would execute by default the main function, but is another's function name is given then it will search for this one to run.

So, module and script seems to me an unnecessary distinction, thus simplifying LFE.

-mg 

You received this message because you are subscribed to a topic in the Google Groups "Lisp Flavoured Erlang" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lisp-flavoured-erlang/5oegsRQqjeU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lisp-flavoured-e...@googlegroups.com.

[Robert Virding]

It depends *why* you want to execute the script. If it just to get something done then it doesn't really matter whether if it a shell script or a module script with a predefined entry point. If, however, you want to initialise your shell when you start it then, of course, a shell script is the only way to go. Just executing functions in a file will do things but it will not allow you to initialise the shell itself.

That's why we cover both: there is lfescript for executing a module by calling the main/1 function; and there is lfe for both executing a shell script and for starting up and being initialised by it. It is primarily the default shell initialisation I was thinking about. What the LFE shell can't do today is both execute something from the command line, either a string or a script file, and then remain in the shell. It is either-or at the moment. This is based on how sh/bash works as I think the way erl handles arguments by default is not very clear.

For both "module scripts" for use with lfescript and shell scripts for use with lfe they can both be used as command files if the the first line is '#! /usr/bin/env lfescript' or '#! /usr/bin/env lfe'. This line is ignored if they are started with the lfescript/lfe command.

A new erlang shell, kjell, has just been released and I am going to check it out and see what we can take from there.

Robert

On Tuesday, February 17, 2015 at 2:36:59 AM UTC+1, Mário Guimarães wrote:

fine :-)

Regarding the module vs script dichotomy, it simply shouldn't exist.

For example, a script could be a module without an explicit module definition, and whose module name is that of the script, and it would not export any function, but when passed to the "lfe" command line this will search for a function named main and execute it, and if the main function does not exist nothing happens and "lfe" ends doing nothing or warns "main not found"

Now for a module to be executed in script mode the "lfe" command just needs to search for a main function and execute it, and this would not even need to be exported!

In fact, the "lfe" command would execute by default the main function, but is another's function name is given then it will search for this one to run.

So, module and script seems to me an unnecessary distinction, thus simplifying LFE.

-mg 

No dia segunda-feira, 16 de fevereiro de 2015, Duncan McGreggor <dun...@cogitat.io> escreveu:

I don't think you missed anything. Your thoughts sound very similar to those that I've had and some other folks' too. I think there's general agreement on these matters and now it's a matter of trying ideas out.

From what I gather, Robert's got a branch where he's working on some of this stuff.

A couple of random notes:

 * I'm not sure how possible it would be to run a module as a script, or treat a script as a module -- these are handled quite differently right now, and may hinge on Erlang internals.

 * Your thoughts on the responsibilities of LFE vs. lfetool are *very* close to what Robert and I have been discussing (possibly identical)

I did read your emails when they came in, but I didn't feel I had anything else to add -- you had stated things well. When Robert has something ready to test, we can try it out and provide feedback on our experiences, and he can then better help us adapt our workflows to what is possible with LFE.

Thanks!

d

On Mon, Feb 16, 2015 at 5:28 PM, Mário Guimarães <mario.luis.guimaraes@gmail.com> wrote:

Hi,
I was expecting some comments to my last two messages ... did I miss something?
-mg

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.
To post to this group, send email to lisp-flavoured-erlang@googlegroups.com.

Visit this group at http://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Lisp Flavoured Erlang" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lisp-flavoured-erlang/5oegsRQqjeU/unsubscribe.

To unsubscribe from this group and all its topics, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.
To post to this group, send email to lisp-flavoured-erlang@googlegroups.com.