Loadable plugins support

290 views
Skip to first unread message

"Sztupák Sz. Zsolt"

unread,
Mar 7, 2012, 10:01:41 PM3/7/12
to ml-d...@googlegroups.com
Hi!

Managed to get a simple, dynamically loading plugin support to work.
This means we can now not only write interpreted lua scripts, but also
compiled C code to control ML and Canon.

So, there are now three separate ways to externally script your
Canon/ML: USB, Lua and now loadable plugins. (Wooo. I'm on a roll. :) )

What better test is for this plugin support to actually put major parts
(= the whole interpreter) of the Lua scripting support to a separate
module. This also means that hopefully lua scripting will work for 550D
users too! (Please test!)

About the implementation:

Plugins are compiled as position-independent, and some linker tinkering
is used to put some section information to the first few bytes of the
binary. Now, the "linker" (strong quotes) will know which sections of
the binary will contain the function and data pointers, that need to be
relocated, and will relocate them. Communication from/to the plugin is
done the following way:

The plugin_export.h file contains all the functions that are exported
from ML/DryOs to the plugin. The ML module loader will send this
information to the plugin, who will load the positions from this
structure, and can use all of the exported functions natively.

The plugin on the otherside can signal the compiler that he wants to
create en exportable function using the EXTERN_FUNC macro. The
definitions of these functions will get to a special place inside the
plugin bin (the almost very start), from where the ML Module loader can
load this information.

The advantage of this solution:
- It's more or less simple. Handling of the exported functions is
actually automated by the macros, so it's not that hard to use.
- You only need to compile the plugin once, and it will work on all
cameras, firmware and ML versions. (if they export the functions you use
of course)

The disadvantage:
- All ML/DryOS functions need to be explicitly exported, for them to be
useable inside the plugins.
- The function export list and function pointers inside the plugins take
away memory.

The other possible way would be to auto-generate stubs files from the
compiled ML autoexec.bin-s and DryOs stubs file, and use them inside the
plugins. The advantage would be that no explicit exporting would be
needed, and all ML/DryOs commands would be available for plugin
developers. There would be also no memory used for the export lists (as
there are none). The major drawback is that each plugin would have to be
compiled for all ML and all Camera versions separately. This idea was
soon discarded.

If you have any other idea on how to implement linking easily feel free
to tell :)

How to build plugins:

The plugins directory contains everything that is needed to build
plugins. Just create a new directory, copy the Makefile from lua or the
testplug directory to your directory, edit it (add the .o files), then
run make, and it will compile. There are some linker scripts and helper
functions available inside the base plugins directory, that will be
compiled in every plugin. After compiling to load them you have to use
the plug = load_plugin(filename) function to load it, and func =
get_function(plug, func_id) or func = get_function_str( plug, func_name
) to get the address of the function. After that func can be used as a
regular function. There is no real API yet to actually use modules (lua
module will be loaded when lua scripting is first used, and the testplug
module is loaded using the "Don't touch" button), but if it is okay with
everyone I'll start some structural changes so external modules can work
as any other part of ML (like extending the "configs" API to allow
adding configuration values later, etc.)

Other questions:

1. I saw that the .bss section is not put inside the autoexec.bin files.
I know it only contains a bunch of zeroes, but wouldn't it be safer if
those zeroes would be explicitly loaded too (they take memory away
anyway)? The plugin makefiles do fill in the .bss section inside the
binary, as during dynamic loading no one can guarantee that the memory
will actually be zeroed out.

2. On 60D I set that the module loader will load the modules inside
"malloc" memory. On all other cameras it will load it inside the
"AllocateMemory" memory. What other cameras are there where "malloc"
memory would be more appropriate?

Binary download link for testing: (I only have a 60D. 550D testers are
very welcome!)
https://bitbucket.org/sztupy/magic-lantern/downloads/magiclantern-2012Mar8.usb.lua.plugin.sztupy.zip.zip

SztupY

Dave Thomas

unread,
Mar 7, 2012, 11:05:59 PM3/7/12
to ml-d...@googlegroups.com
I have a 550D, I just read your post on ML Controller from last week, http://groups.google.com/group/ml-devel/browse_thread/thread/4b0d01d64a459485/5a9bfd36701fcbec?lnk=gst&q=Sztup%C3%A1k#5a9bfd36701fcbec

How exciting, I'm going to try all this out now!




--
http://magiclantern.wikia.com/

To post to this group, send email to ml-d...@googlegroups.com
To unsubscribe from this group, send email to ml-devel+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/ml-devel?hl=en

Daniele Musso

unread,
Mar 8, 2012, 12:04:25 AM3/8/12
to ml-d...@googlegroups.com, ml-d...@googlegroups.com
Wow! I think you can add that it will be easier to isolate/find bugs !
What about performances/startup speed?

Great work !

Inviato da iPhone

Il giorno 08/mar/2012, alle ore 04:01, "Sztupák Sz. Zsolt" <ma...@sztupy.hu> ha scritto:

> SztupY

Redkite Bart

unread,
Mar 8, 2012, 3:29:10 AM3/8/12
to ml-d...@googlegroups.com
Great stuff. Well done.

Maybe this can lead to a basic stable ml core with all the common features like zebra, peaking, iso, kelvin, timelapse, audio followed by a few plugin tabs where you can add plugins with additional functionality.  Kind of a ML appstore.

Redkite Bart

unread,
Mar 8, 2012, 3:35:31 AM3/8/12
to ml-d...@googlegroups.com
Or even just a ml kernel with a mapstructure for the menu's and single file plugins. So every menu tab has a map on the sd card and you just put single file plugins in these folders and they appear as features in the menu's.

eggnot

unread,
Mar 8, 2012, 5:43:25 AM3/8/12
to ml-d...@googlegroups.com
IT'S ALIVE!
i mean it works on 550d. wb_shoot.lua tends to glitch(shoots reapiteadly, trigged with play button for some reason, in M mode not sleeping for 1000, etc), test it more later. it's small bugs i bealive to be eliminated.
 
THE BIG THING, - YOU ROCK'S MAN!
 
if it will work, i really think it's great milestone in ML development in general.
modular system is looks good. scripting is awesome.
 
the only way to control canon is missing for now is by brainwawes ~~~:-)
 
GREAT WORK, again. keep going.
 
--
 
четверг, 8 марта 2012 г. 10:01:41 UTC+7 пользователь SztupY написал:
CONSOLE.LOG
DRAM.BIN

eggnot

unread,
Mar 8, 2012, 6:12:57 AM3/8/12
to ml-d...@googlegroups.com
ok for now:
 
i mean it works on 550d. wb_shoot.lua tends to glitch(shoots reapiteadly, trigged with play button for some reason, in M mode not sleeping for 1000, etc), test it more later. it's small bugs i bealive to be eliminated.
 
 
glitch was because of changing from video mode to M-mode while wb_shoot.lua was running.
i guess we need some button(long press maybe?) to terminate script. while running it's canceling menu navigation.
some icon or indicator of running script maybe too?
 
all other thing works well, stability test passed ok.
 
GREAT JOB, once again.

Giovanni Di Cello

unread,
Mar 8, 2012, 7:48:33 AM3/8/12
to ml-d...@googlegroups.com
I can test stuff on my 50D, let me know if you know how to do it.
;)

"Sztupák Sz. Zsolt"

unread,
Mar 8, 2012, 9:07:40 AM3/8/12
to ml-d...@googlegroups.com
2012.03.08. 13:48 keltez�ssel, Giovanni Di Cello �rta:

> I can test stuff on my 50D, let me know if you know how to do it.
> ;)
> --
> http://magiclantern.wikia.com/
>
> To post to this group, send email to ml-d...@googlegroups.com
> To unsubscribe from this group, send email to
> ml-devel+u...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/ml-devel?hl=en
Just install on the camera and try to run a script with it.

SztupY

Giovanni Di Cello

unread,
Mar 8, 2012, 12:30:32 PM3/8/12
to ml-d...@googlegroups.com
I thought it was for 60D only, my bad.
Testing in a while :)

Giovanni Di Cello

unread,
Mar 8, 2012, 12:59:30 PM3/8/12
to ml-d...@googlegroups.com
Scripts work ;)

Lauris Heijdemann

unread,
Mar 8, 2012, 3:49:53 PM3/8/12
to ml-d...@googlegroups.com
Thanks !!!!!  "Sztupák Sz. Zsolt"

Perhaps now we can program the canon to roll on a dolly, pan, tilt, zoom, focus.
By pulse wide modulating the led or by beeping via monitor audio?
Reply all
Reply to author
Forward
0 new messages