load('#!/usr/local/bin/lua')()

[Martin Eden]

Hello list,

Tonight I want to discuss shebang support.

test.lua
[
  #!/usr/local/bin/lua
  print(42)
]

Issues:

1. Out of scope

  That first line is not Lua.

  It's Unix shell hack to run command in first line for the rest of file.

2. Inconsistent

  There is no function to compile that file contents as string:

    > local SourceCode = '#!/usr/local/bin/lua\n print(42)'
    > load(SourceCode)()
    < stdin:2: attempt to call a nil value

  However we have two global functions just to work around this issue:

    > dofile('test.lua')
    < 42
    > loadfile('test.lua')()
    < 42

  I bet both of them will ignore first line if it starts with lattice.

Questions:

  My questions mostly to Roberto and lhf, but community opinions are
  nice to hear too.

  1. What's your vision of this feature? (Do you wish it would never
    been in language?)

  2. Can load() support shebang?

(
  I'm hitting this as my Lua code melder embeds source code as string.
  I want to preserve source code as it is. So the only today's option
  to load code with shebang is strip first line if it starts with "#"
  before calling load(). This looks weird from my point of view.
)

-- Martin

[Sainan]

I'd say it's a bit of a weird inconsistency that 'loadfile' and 'dofile' skip the shebang while 'load' does not.

But, it's also kind of your problem now, because you're reading a file specifically to 'load' it later, so you need to replicate this behaviour of skipping the shebang at some point before 'load' is called.

-- Sainan

[Vinícius dos Santos Oliveira]

Em qua., 27 de nov. de 2024 às 02:03, 'Sainan' via lua-l
<lu...@googlegroups.com> escreveu:

> But, it's also kind of your problem now, because you're reading a file specifically to 'load' it later, so you need to replicate this behaviour of skipping the shebang at some point before 'load' is called.

Line numbering for debug info (e.g. raised exceptions) will be wrong
if load() itself doesn't handle this problem.


--
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

[Sainan]

An empty line is still valid Lua code last I checked. ;)

-- Sainan

[Christophe Delord]

Here is what LuaX does to make scripts loadable with load:

function comment_shebang(script)
    return script
        : gsub("^#!.-\n(\x1b)", "%1")   -- remove the whole shebang of compiled scripts
        : gsub("^#!", "--")             -- comment the shebang before loading the script
end

Just call load(comment_shebang(script)) instead of load(script) to load a script, ignoring the shebang and preserving the line numbers.

&#8203;

[Roberto Ierusalimschy]

>   I bet both of them will ignore first line if it starts with lattice.

You don't need to bet:

"The first line in the file is ignored if it starts with a #."
https://www.lua.org/manual/5.3/manual.html#luaL_loadfilex

>   1. What's your vision of this feature? (Do you wish it would never
>     been in language?)

Luckily, it never was part of the language. As you said, it is a
hack in loadfile.

>
>   2. Can load() support shebang?

Then it would be part of the language, and probably I would whish
it wasn't. :-)

> (
>   I'm hitting this as my Lua code melder embeds source code as string.
>   I want to preserve source code as it is. So the only today's option
>   to load code with shebang is strip first line if it starts with "#"
>   before calling load(). This looks weird from my point of view.
> )

Shebang files are not Lua scripts, they are shell scripts. If you want
to support shebang files, you need to add support for that in your
program, like loadfile does.

BTW, loadfile has another similar hack for Windows, that it skips UTF-8
BOM sequences at the start of a file. If you want to run Lua scripts
with an added UTF-8 BOM sequence without using loadfile, you will also
have to remove it explicitly.

-- Roberto

[Martin Eden]

On 2024-11-27 20:24, Roberto Ierusalimschy wrote:
> Shebang files are not Lua scripts, they are shell scripts. If you want
> to support shebang files, you need to add support for that in your
> program, like loadfile does.

Thank you for clarification! For some reason I believed I saw them
in language grammar for <chunk>, but didn't find it in documentation
today.

> BTW, loadfile has another similar hack for Windows, that it skips UTF-8
> BOM sequences at the start of a file. If you want to run Lua scripts
> with an added UTF-8 BOM sequence without using loadfile, you will also
> have to remove it explicitly.
>
> -- Roberto

Yes, possible binary bytes prefix is another trap when loading
file as text. 0x(EF BB BF) for UTF-8, and 0x(FF FE), 0x(FE FF) for
UTF-16 little- and big-endians. I think I stepped on that when
writing Lua code formatter. Quite annoying to meet them in
7-bit ASCII file.

-- Martin

[Martin Eden]

On 2024-11-27 19:14, Christophe Delord wrote:
> Here is what LuaX <https://github.com/CDSoft/luax> does to make

I think the sane approach to load Lua shell script as Lua string is
convert shebang #! to Lua's line comment "--#!".

But we will still lose ability to convert that string back to what
it was:

  [#! blarg] -> [--#! blarg] -> [#! blarg]
  [--#! Sir please help me !] -> ditto -> [#! Sir please help me !]

We need to store additional information about that file string.
(And for shebang hack to work file needs "+x" executable attribute.
So we'll need to store "x" attr too.)

That complicates design and annoys me.

I think I'll just pretend shebang hack never existed.

Supporting features you don't like spoils all the fun of creation.
Even if that features is your own previous design decisions.

-- Martin

[Gé Weijers]

On Wed, Nov 27, 2024 at 8:15 AM Vinícius dos Santos Oliveira <vini.i...@gmail.com> wrote:

Em qua., 27 de nov. de 2024 às 02:03, 'Sainan' via lua-l

Line numbering for debug info (e.g. raised exceptions) will be wrong
if load() itself doesn't handle this problem.

 

Replace the shebang with '--', this will comment out the first line but will keep the line numbers the same.

This seems to work just fine:

local BLOCKSIZE <const> = 256

local function load_my_file(fn)
    local f <close> = assert(io.open(fn, "r"))

    local first = true

    local function get_block()
        local block = f:read(BLOCKSIZE)
        if block and first then
            block = string.gsub(block, "^#!", "--", 1)
            first = false
        end
        return block
    end

    return load(get_block, fn, 't', _ENV)
end

--

Gé

[Frank Dana (FeRD)]

On Wednesday, November 27, 2024 at 3:55:40 PM UTC-5 Martin Eden wrote:

I think the sane approach to load Lua shell script as Lua string is
convert shebang #! to Lua's line comment "--#!".

 

We need to store additional information about that file string.
(And for shebang hack to work file needs "+x" executable attribute.
So we'll need to store "x" attr too.)

...Why? The execute bit is meaningless to Lua, so you'd only need to keep track of it if you want to, say, recreate the input file somewhere else with its filesystem permissions intact. And if you're going to do that, you should also keep track of the owner and group from the original file, the extended filesystem xattrs from the original file...

It's also worth noting that losing execute bits from duplicated files is sometimes considered a feature, as it's safer if scriptable files aren't marked executable unnecessarily, slightly raising the barrier to executing their payload without sufficient scrutiny. GitHub Actions, for example, strips the execute bit from every file it publishes as a build artifact, so that any potentially malicious executables downloaded by overly-trusting users at least take a bit of extra effort to set loose on their computers.

[Martin Eden]

On 2024-12-02 04:37, Frank Dana (FeRD) wrote:
> ...Why? The execute bit is meaningless to Lua, so you'd only need to keep
> track of it if you want to, say, recreate the input file somewhere else
> with its filesystem permissions intact. And if you're going to do that, you
> should also keep track of the owner and group from the original file, the
> extended filesystem xattrs from the original file...

Yeah, I'm going to write "unmelder" for code produced by packer.
Just because it can be done.

Yes, file owners and permissions. And creation and modification and
access times... Well that's what archive programs do. When main focus
is just create files and directories with correct names, recreating
attributes is boring and not really important.

In the end of that message I've announced that I'll ignore shebang.
So packed code with shebangs will give compile-time error.

Point of starting thread was to claim that shebang hack is too hard
to support and probably not worthy to be supported at all.
("#! Considered Harmful" ;) ).

-- Martin