Interrupt Callback Functions for small IoT systems (small ROM-RAM)

[bil til]

Sorry, if this "wischful thinking" of me is a bit a "heavy wish"... .

For single chips IoT systems with small microcontrollers, e. g.
STM32G0 with ROM up to 512kB and RAM up to 128kB, I am extremely happy
now that in Version 5.5 the bin files use "function start" aligning on
32bit. I am still on other "construction sites" and could not test
this yet, but I look forward to this test very much.

Just I have one hard wish, which currently in Lua still is impossible.
I have no way in Lua to give the user a possibility to program a Lua
Callback function, which would interrupt the Lua task scheduler for
some "short actions".

The main problem here is the address of the callback function. As in
Lua Functions generally are Objects in the Global table, for full
flexibility before a functino is invoked, this must be looked up in
the global table... just this is much too much fuzz for an interrupt
function (and this would also collide with the regular Lua work
currently running when an Interrupt arrives.

BUT in my ROM based Lua Code now: Here the Functions, also the User
Lua code is ALL fixed. This callback function of course must NOT be a
RAM based function which has been programmed dynamically during
runtime. But ONLY functions in the pre-programmed BIN code (residing
in ROM) will be allowed to be used for such callback.

And these ROM functions OF COURSE are fixed address functions... .

The condition that this would work, would be just that there would be
some C function "Lua_getConstFunctionAddress" or similar... . And of
course this must be FAST.. Best would be, that this
"Lua_getConstFunctionAddress" is in fact more some
PreCompiler/Macro-Function, and in the BIN code somehow DIRECTLY the
constant ROM Function code would be used.

.Would this perhaps be a way to go?

The next problem would be, that the current LUA workforce state must
be pushed to some stack and later retrieved. (this could be a special
"Interrupt heap" - of course the code in such interrupt functions can
be required to be very restricted - for VERY many useful applications,
it woudl be already fine if some integer assignments are possible and
some library functions are called, which then do the remainder in C
without any Lua access).

[Martin Eden]

Hi Bil,

Sorry if I misunderstood but you want to have interrupt handler
as Lua function?

Besides Lua I'm tinkering with embedded too. But I'm on ATmega328.
That's 2 KiB RAM and 32 KiB ROM.
(https://github.com/martin-eden/Embedded_Crafts/tree/master/Parts)

You can write generic interrupt handler like

  (
    Const OurId = 42
    Call ( GlobalFunctions[OurId] )
  )

"GlobalFunctions" is array of pointers to ROM.

This way you can change handler to one of hardcoded ROM addresses
at runtime.

But this costs memory and several ticks. Worse, you can't really do
much from interrupt handler. You'll see machine in frozen state.
Inconsistent program data and pending interrupts with lower priority.

So most you can do is get current time and store event for further
processing in main program code. Why use Lua for that?

-- Martin

[bil til]

Thanks, this sounds interesting.

Though I meanwhile switched to another idea with events... (but
currnently still in "idea state", not yet finally implemented :) ) .

On Wednesday I had a hard day in trying to run MicroPython on my
controllers for comparison, as MicroPython says that they clearly
support "scripted" Interrupts... .

MicroPython has of course the advantage, that it is really a meanwhile
VERY dominant and popular programming method.

I was happy when I succeeded to get the source code compilation
without errors after a "hard time" Thursday, but after I gave up the
ambition to get it done by my loved Keil ARMCC and I used their gnu cc
plus MSYS2 environment for Python3.exe plus make.exe, it really
worked, also generating this "mystical" QSTR header file
automatically, which really seems to require GNUCC for automatic
generation.

... so far so "quite good"... .

In the evening then I spent some time looking at MicroPython further -
I understand that the programming style really is quite nice, I
especially liked this print function "print( ..., [optional sep=...],
[optional end=...]).... this seems to be stunningly nice in my eyes,
especially for "lazy time to time programmers", which user programmers
often are... .

But then arrived the hard BUT: The async library for cooperative
multitasking seems to be a terrible mess... I can not imagine, how a
"simple user" without much programming experience could use this for a
generating a useful and working program - further in the docs it says,
that this is still in "Beta state"... and furhter there are quite a
bit of warnings in Dr. Google, that in the large Python3 the
pre-emptive multitasking seems to be "programmed buggy", so there is a
warning to use this for secure programs / or even an urgent
recommendation to test this extremely carefully.... .

On Wednesday morning I had announced to my crew already, that they
should arrive the next day in black clothes as we might bury Lua...
but on Thursday morning after "sleepoing over this Micropython async
problems", I reufully but also of course very glad returned to Lua...
:).

Concerning Scripting applications for IoT hardware control, there are
I think three clear "difficult to implement" but very necessary
implementation requirements:
- it must compile for restriced ROM RAM. The 256kB ROM / 20kB RAM
requirement for Micropython is acceptable in time of currently
available controllers, but the 80kB ROM / 20kb RAM of my current Lua
version of course is better (also in Lua it is much easier to strip
out quite a bit of "heavy functions", as they are mainly all lib
based, and it is easy to strip the libs... MicroPython has much more
functions "as is" ... I think this can in no way be stripped down
further - Lua currently is a factor 3-4 more ROM efficient - my
minimum ROM controllers have 512kB - so also ROM quite sparse, as it
also must include the user scripts...)
- it must support cooperative multitasking in some easy and clear way,
also for "simple minded time-time programmers". MicroPython (and even
"full Python") in this aspect with this async lib seems to be
non-acceptable in my sight. In Lua also I do not like so much the
"coroutine solution", but in Lua it is straight away to remove the
corountine library and implement own task handling with resume / yield
- this is super nicely documented in Roberto's Programming in Lua book
and also programming is very straight forward.
- it must support interrupts programming in some way. Here I like the
"fix function adressing" of MicroPython - so I can imagine it
"somewhow" will work there as thtey proclaim (although therre are
warnings in Dr. Googgle, that it is generally a stupid idea, to
implement interrupts by scripting...).

... so now I will implement interrupts by "Event classes", and the
user program then has to ask the events in some thread to get the
itnerrupt infos... , best of coures in the main thready, but for
"lower priority actions" also tin the slower tasks - this then is up
to the user.

I hope that this works fine and I will report how this works when I
did it and tested the it carefully... . I am now somehow very excieted
to come back to Lua programming in the next weeks, I then will also
test the "keep bin code in ROM work" in Lua 5.5 - I hope then I get
this run quite easily with minimum Lua software code changes... . (as
the bin format now is modified such, that the functions all start 32
bit aliinged in respect to the file start, this should work).

Am Sa., 4. Okt. 2025 um 04:35 Uhr schrieb 'Martin Eden' via lua-l
<lu...@googlegroups.com>:

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/lua-l/79e3dcf7-8dba-40d4-abdb-d36c5df20535%40disroot.org.

[bil til]

PS: One further idea concerning this named function parameters, which
are somehow stunningly useful in the print(..., [sep=...], [end=...])
application which really for quite a bit of "somehow often used
formatting tasks" is stunningly easily solved in Python programming.

Now I think of adding two functions "printsep( sep)" and "printend(
end)" in Lua, which would modify just the NEXT print command... . (and
after this it will reset to the standard print settings with space for
sep and \n for end).

So e. g. for Python command print("Hello world", end=""), In my Lua
the user could program "printend("");print( "Hello world'").

Or for Python command print("Hello", "world", sep="|"), In my Lua the
user could program "printsep("|");print( "Hello", "world'").

At least from first view this looks nice. Just I have to test it in
detail first. This might become, if the print function does not want
to use a static buffer, but wants async sleep for giving time to other
tasks during slow serial communcation ... .... but I think this can be
fixed - I will see... .

[Francisco Olarte]

On Sat, 4 Oct 2025 at 15:49, bil til <bilt...@gmail.com> wrote:

PS: One further idea concerning this named function parameters, which
are somehow stunningly useful in the print(..., [sep=...], [end=...])
application which really for quite a bit of "somehow often used
formatting tasks" is stunningly easily solved in Python programming.

Now I think of adding two functions "printsep( sep)" and "printend(
end)" in Lua, which would modify just the NEXT print command... . (and
after this it will reset to the standard print settings with space for
sep and \n for end).

So e. g. for Python command  print("Hello world", end=""), In my Lua
the user could program "printend("");print( "Hello world'").

Or for Python command print("Hello", "world", sep="|"), In my Lua the
user could program "printsep("|");print( "Hello", "world'").

At least from first view this looks nice. Just I have to test it in
detail first. This might become, if the print function does not want
to use a static buffer, but wants async sleep for giving time to other
tasks during slow serial communcation ... .... but I think this can be
fixed - I will see... .

Have you considered some easier alternatives?

I.e. you can do 

local DPO = { sep="", end="" } -- Default Print Options.

local function PO(opts, ....) 

 --- Do whatever using (opts and opts.sep) or DPO.sep as separator.

 --- Or alternatively, do opts=opts or {} at start.

 --- Or get rid of DPO and use opts.sep or "" ( the nice thing of 

 --- DPO is you can publish function, like printsep, to alter it

 --- in a checked way.

end

-- And then:

local function Pdefault(...) return PO({},...) end

--- Tail call optimization makes Pdefault cheap.

--- If you do not plan on supporting printing tables then you can do:

local function Pauto(maybe_opts, ...) 

   if type(maybe_opts)=="table" then

     return PO(maybe_opts, ...) 

   else

     return Pauto(maybe_opts, ...) -- tail call everywhere.

   end

end

--- Of course, if you plan to support printing tables with metamethods or similar

--- you still can use PO/Pdefault for that cases.

--- You can put options at end and use select, but in lua it is more natural 

--- at the start.

--- Now export things from your module if you want.

return {

  print_default=Pdefault, print_with_opts=PO, print=Pauto

}

Not as good as python support, but equally useable, and it is probably much easier to implement and faster than your "manipulators" ( functions like your printend/printsep are called manipulators in C++ ) and more easy to do in a thread-safe way (even if you are talking of threading by coroutine switching, manipulators need careful sincronization, this solution is easier to synchronize ).

Francisco Olarte.

[bil til]

Thanks for recommendation.

But how would the "10 years old users who seem to love Python" invoke this?

I hope that my solution does NOT require task sleeping - in this case
my solution would be straight forward and (I think) as easy to learn
as the python print stuff... .

(I do NOT want that the users have to invoke/ remember different
function names for this application, as e. g. "print" and
"printdefault" - this of course also would be easy to implement, but
this would strongly "pimp up" complexitiy. (in principles you then
need 4 functions: print, printse, prints, printe - and all use
different first parameters...).

Or did I not understand you correctly, and you want to use a 2-par
table as first print par for every invodation of print? ... this of
course also would be implemented easily, but this in my ears for
"simple mindes users" sounds very complex... . Arrays are concepts,
which to my experience wiith our electronics trainees are too complex
for "newbies in programming".

Am Sa., 4. Okt. 2025 um 18:35 Uhr schrieb Francisco Olarte
<fol...@peoplecall.com>:

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/lua-l/CA%2BbJJbzRxa_urzEskv6QhUwDG091_j4s%3DfutV5OhcX-%3DyefaFA%40mail.gmail.com.

[Francisco Olarte]

On Sat, 4 Oct 2025 at 23:25, bil til <bilt...@gmail.com> wrote:

But how would the "10 years old users who seem to love Python" invoke this?

Do you really mean "10 years old users" or "people using python for 10 years"?

I hope that my solution does NOT require task sleeping - in this case
my solution would be straight forward and (I think) as easy to learn
as the python print stuff... .

Well, IMHO a leading/trailing table seems to be more similar. Manipulators need to be leading anyway, so there you have a mismatch with pythonic ways, matching the more easy/luaesque leading table, and I think 

print({sep="x"},...) is more similar than printsep("x");print(...), but anyway, your code, your style.

And I do not know what you mean by task sleeping. If you are talking about my synchronization comments, I was thinking of something like several of my codes. They make heavy use of coroutines, which means with my single function approach you only need to insure PO does not context-switch ( or, if it does, you can reduce it to a preformatting into a buffer and then send it out with a single call to a non-switching function ). Whereas your manipulator-like approach needs to think on the case:

    printsep(..);code_which_yields();print()

and then code_which_yields makes a print and destroys your seps ( and prints badly because it uses it ).

Seems difficult but think of the case:

printsep(","); print("Result:", calc_res());

And calc_res manages to yield.

Even worst, calc_res() may be a non yielding function which sometimes prints.

The table passing solution avoids this cases. I normally tend to protect from this cases as I have found this kind of aux code gets used a lot and this things bite you years later.

 (I do NOT want that the users have to invoke/ remember different

function names for this application, as e. g. "print" and
"printdefault" - this of course also would be easy to implement, but
this would strongly "pimp up" complexitiy. (in principles you then
need 4 functions: print, printse, prints, printe - and all use
different first parameters...).

But your solution means they need to remember the manipulators. That, in my case, was the cause I used C printf instead of streams in C++ until std::format and friends came around.

You do not really need many functions, if you make the options be the first parameter when it is a table you can use Pauto trick I wrote, and just have to remember to put an empty table in the normally very strange case of Pauto({}, table_which_prints_via_tostring_or_similar, ...)

 Or did I not understand you correctly, and you want to use a 2-par

table as first print par for every invodation of print? ... this of
course also would be implemented easily, but this in my ears for
"simple mindes users" sounds very complex... . Arrays are concepts,
which to my experience wiith our electronics trainees are too complex
for "newbies in programming".

I do not use a 2-par table, I use an (optional) map. If you read a little I have a base function, PO for print_options(parameters_map, ...data_to_print), then I do a Pdefault fo print_using_defaults which just passes an empty parameter map to PO. Then build a Pauto for print_autodetecting_parameters which switches on the first parameter type and calls either one, then suggest to export them renaming as needed ( this is a common pattern in my code, short local function names in the module, where you have no trouble remembering which is which, export them with more meaningful names, let user require & rename to shorter ones if needed ).

For newbies this is not usually too complex, just call Pauto ( you can rename it as print ) and put an optional table at front. I do not normally like this, I have found over the years the conceptual charge of this "overloading" is greater than that of having two functions, so these days I will normally export PO as printX or print_with_opts and Pauto as print, but 30 years ago I would probably have gone the overload way.

There are a thousand ways to do it, you can even do something like..

function OPTS(t) return function(...) return PO(t,...) end end

export that as print_with_format and then do

print_with_format{sep=",", end="\n"}(...) 

But it is just cosmetic sugar ( having the braces outside the parentheses )

It is just a matter of taste. Lua is a simple language, it does not have the extremely complex parameter passing ways of python. But python has everything you can imagine and the kitchen sink thrown in, I have done really bizarre stuff (ab)using the classes, metaclasses, meta-meta classes, parameter passings and other stuff.

Francisco Olarte.

--- Uninteresting bottom quoted stuff left for reference.

 

To view this discussion visit https://groups.google.com/d/msgid/lua-l/CAOnDXmHfZHz3o2gnwKw1RfxFDTUVqhz8newRQzEkN5jX6iSupA%40mail.gmail.com.

[bil til]

Thanks for detailed answer.

The "Learning programming with python for 10 year olds" is just a "Dr.
Google" answer, sorry, for this - maybe a bit "not really
qualified"... .

Googling for "Micropython demo" in (German) Google, a nice 10 min demo
is shown in YouTube "MicroPython & ESP32 - Grundlagen der
Programmierung" (sorry in German, I tried to find something similar in
English, but did not work...). In Minute 7-10 there they show an (to
my impression) extremely impressive and "easy-coding" introduction
into print in MicroPython, this stunned me a bit due to its
simplicity... .

You are completely right, that generally "option forwarding" by such
"variable transfer" is potentially dangerous and non-professional... .
But to keep it easy for "simple minded programmer" I do not see
another option.

Your warning example

printsep(","); print("Result:", calc_res());

of course is clear - if calc_res includes any sleep / yielding / task
switch, this printsep will fail... .

I anyway like it too much due to simplictiy... . ..but I will see...
.This topic is not VERY important for me... . It is more just a "side
gimmick" ... I will see whether I still will like this in 2-4 weeks
:).

Am So., 5. Okt. 2025 um 12:38 Uhr schrieb Francisco Olarte

> To view this discussion visit https://groups.google.com/d/msgid/lua-l/CA%2BbJJby9D7A9K6aZAaoZV6W_MkmV_VXbY9qrKSPHMow19hR9Kw%40mail.gmail.com.

[Sewbacca]

I think it's worth pointing out you could use the fact that tables are associative arrays and bundle arguments and options in a single elegant table call, which allows to skip the extra pair of brackets if the table is the only argument:

print { first, second, sep=', ' }

Its important to note that if you want to expand multiple arguments, the unpacking function must be added after any options and arguments:

print { first, second, sep=', ', table.unpack(moreargs) }

otherwise the rest would get discarded.

You could then access positional arguments via integer indicies and parameters via names

function print(t)
     io.stdout:write('first arg: ', t[1], '\n')
     io.stdout:write(table.concat(t, t.sep), '\n')
end

[Sainan]

How are we suddenly talking about named arguments again? Do not create a table for each function call, this an absolute anti-pattern for performance.

-- Sainan

[bil til]

... sorry sorry... .

In our discussion some weeks ago I also thought that the named
parameters really are somehow a "not really necessary side gimmick".

I did not see, how VERY nice this is for the VERY special application
of a function like print(...) (with ONLY optional arguments, plus not
any "argument type" restrictions...). ... this really is very special
case ... just when I saw this MicroPython youtube demo, I have to
admit, that I was somehow very impressed, how useful this can be
applied for this special case... .

But I think I will be happy with my printsep / printend .... I will
report you in 3-4 weeks I hope :).

But concerning "implementing named parameters" as new "language
gimmick" I tend more on the conservative side... . Changing basic
language scripting rules always tends to nerve VERY many people, it
also will e. g. collide with very many nice Lua code editors... .

Am Di., 7. Okt. 2025 um 12:17 Uhr schrieb 'Sainan' via lua-l
<lu...@googlegroups.com>:

>
> How are we suddenly talking about named arguments again? Do not create a table for each function call, this an absolute anti-pattern for performance.
>
> -- Sainan
>

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/lua-l/t5NibNCBeF55zhWj0vLoPieKnN5iCSg8Q46phMPJpA5VUNYqg6H5DZOmEw1PKehRGp7VWV44N3DduhhFJvVLp7Neq7MASGn6cxBUnAWSo28%3D%40calamity.inc.

[Javier Guerra Giraldez]

On Tue, 7 Oct 2025 at 05:17, 'Sainan' via lua-l <lu...@googlegroups.com> wrote:

How are we suddenly talking about named arguments again? Do not create a table for each function call, this an absolute anti-pattern for performance.

that is not a good general advice.  there are countless applications where it can be the best option.  it's at least worth a try.

- if the call does any kind of I/O, creating a small table will take an insignificant amount of time.

- if the large call initialises an object that will be used lots of times, it will be amortised over.

- sometimes the called function will create several tables anyway.  one more might not be an issue.  maybe the passed "args" table can be used directly, so there's no "extra" table creation.

- in some implementations (at least LuaJIT), a table expression in the source code is stored as a template in the bytecode, making the runtime creation much faster (just alloc+memcpy+store new values at constant slots)

- python actually does it.  yes, it creates a custom dict object for each and every call of a function with named parameters.  and the original post mentioned how microPython performance is ok for this exact use.

 

--

Javier

[bil til]

concerning print function, I think performance / speed is not crucial,
as it practically always is a very slow "communication braked"
function... .

In the "named parameter" version of this Micropython print I was just
somehow stunned, how "easy going" and flexible this can be applied by
"newbee programmers" to create quite fancy output possibilities, which
in any other languange would implement quite a bit more effort /
programming code / programming complexity... .

Am Di., 7. Okt. 2025 um 15:40 Uhr schrieb Javier Guerra Giraldez
<jav...@guerrag.com>:

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/lua-l/CAFkDaoR-4R20CxX7%3Dz8HcZPfyJG3y%3DHx%2BHg_kWR%3DvM4prHvHnA%40mail.gmail.com.