Adding a scripting language

[David Lutterkort]

Hi,

yesterday, I set out to quickly add a provider for ssh_authorized_key - seeing how I know Augeas fairly well, I figured this would be a quick one. Unfortunately, this set me off on an unhappy journey: as it turns out, shell and augtool aren't really a great combination for this (there's just a little too much data munging that would need to happen to make this still comfortable in shell) I reckon it would actually be quicker, maybe a couple of hours, to write such a provider in C++, but that is sorta counterproductive as one of the libral goals is to _not_ force people to learn C++.

Doing this in pretty much any scripting language would also be really quick, but that begs the question of which one, and whether it's reasonable to expect, say, Python or Ruby to be installed - it seems a rather hefty requirement for something as mundane as twiddling lines in some file.

That made me think about adding a scripting language to libral; seeing how I want to keep its footprint small, the choices seem to be Lua, Duktape (Javascript), or mruby. I feel that asking people to learn lua or javascript for writing providers is a bridge too far, especially since our core audience will likely not be familiar with them. The interpreters for them are impressively small (<300k for Lua and Duktape), but also very bare-bones when it comes to support for doing common things like access the filesystem or make web requests.

I therefore started looking at mruby a little more seriously, and actually really like it. It's a bit bigger than the other ones (~ 700k) but much much easier to extend as it has a system of addon libraries that can be built into it - the default mruby build is also very barebones, but adding some mrbgems to support IO, File, Process, JSON handling, and 'require' (yes, not in mruby by default) seem to give a reasonable environment without breaking the bank.

The main downsides I can see are

• the way mruby runs its community is lacking a lot of transparency
• the Ruby dialect is Ruby 1.9
• no 'normal' gems
• it's slow (the benchmarks that come with mruby run 5-10x slower with mruby than with MRI 2.3.1; that's roughly comparable with MRI 1.8.7)

At the same time, it feels like an attractive option to add out-of-the-box scripting that won't be foreign to users, keeps the footprint small, and is reasonably full-featured. mruby is also made for embedding (something that's not practical with MRI Ruby) and it's pretty easy to spin interpreters up and down.

One minor question around all this is whether mruby should be embedded in libral outright, or just be made available as an external interpreter. For now, I'll play some more with it, look into adding it to the libral build process, and try to write some providers with it.

I'd love to hear other people's thoughts, especially if you've ever used (or looked at) mruby before.

David

[Andy Henroid]

David, mruby looks great. As for embedding it or using it as an external interpreter, I'm really split-minded on that question. Embedding would simplify packaging and distribution for libral; mruby would always be there in the binary and at the expected version. However, the external interpreter model might also be good, allowing us to leave mruby behind when not needed or wanted, like barebones container environments. I'm sure you've have the same thoughts.

Andy

--
You received this message because you are subscribed to the Google Groups "libral" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libral+unsubscribe@googlegroups.com.
To post to this group, send email to lib...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/libral/CAHN%2BA%2BX_vVbC7-aKMKbrE1OXzmu%3D%2BXjkeBi3vRdPhfgMRcZ6CA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[David Schmitt]

Ain't you confusing packaging (making a dependable version available) and run time environment (in process vs separate process)?

Regards, David

To view this discussion on the web visit https://groups.google.com/d/msgid/libral/CAJ2H4QORfO2Uzm1xTVxEkCi66VHEMUvJiBSno9563erbC15g7w%40mail.gmail.com.

[David Lutterkort]

On Fri, Mar 3, 2017 at 10:11 AM, Andy Henroid <andy.h...@puppet.com> wrote:

David, mruby looks great. As for embedding it or using it as an external interpreter, I'm really split-minded on that question. Embedding would simplify packaging and distribution for libral; mruby would always be there in the binary and at the expected version. However, the external interpreter model might also be good, allowing us to leave mruby behind when not needed or wanted, like barebones container environments. I'm sure you've have the same thoughts.

Yes, that's pretty much how I am thinking about it: if it's embedded, it can maintain state in between invocations of the provider (say, between listing resources and then updating them) If it's external, it's easy to leave behind.

The other big question in my mind is if it's possible to provide enough of the Ruby stdlib that it provides a runtime that is comfortable enough for most uses. (And not kill ourselves in the process) That runtime question comes in two flavors: there are some basic things, like Etc, that I think need to be there that we'd have to port over from MRI. It's not a monumental task - I spent a few hours yesterday porting the Augeas bindings over; it was tedious but not a huge undertaking. The other is that some of the things you'd want are pretty big, for example, mruby-curl, mostly thanks to libcurl's and openssl's sprawling dependencies.

David

[Andy Henroid]

DavidS, good point, my comment was conflating packaging and runtime. DavidL's question was, I believe, centered on runtime environment (though eventually packaging should be discussed as well) and the benefit I stated for the "external" model stands: easy to decouple and run in lightweight environments. The other benefit I listed ("mruby would always be there in the binary and at the expected version") could apply to either model depending on how the implementation is packaged.

Andy

To view this discussion on the web visit https://groups.google.com/d/msgid/libral/CALF7fHZW3YDyBpQLmuWNC1iv_4zT2HeNjquv2v2wfyY8GyYkCQ%40mail.gmail.com.

[David Lutterkort]

If anybody is interested, I just put up a PR that adds mruby - there are a couple rough edges in terms of setup/installation, but other than those, things seem to work. I am starting to like this mruby thing more and more ...

The PR also has a small bridge between current libral API and (some approximation) of the t&p proposal from puppet-dev - that bridge should go away when I rework the libral API.

David

To view this discussion on the web visit https://groups.google.com/d/msgid/libral/CAJ2H4QOnZHFCshdMqN8bX7H2nh4J6tAGHmPRgJZzwmyFYtUj-g%40mail.gmail.com.

[David Schmitt]

On 5 March 2017 at 05:36, David Lutterkort <lut...@puppet.com> wrote:

If anybody is interested, I just put up a PR that adds mruby - there are a couple rough edges in terms of setup/installation, but other than those, things seem to work. I am starting to like this mruby thing more and more ...

The PR also has a small bridge between current libral API and (some approximation) of the t&p proposal from puppet-dev - that bridge should go away when I rework the libral API.

If you're referring to Ral::CLI in https://github.com/puppetlabs/libral/pull/9/files#diff-6a615615c290551b1800bc22d3a4bf34R108 , I couldn't find the code for that. Did you add it to the commit?

Cheers, David

[David Lutterkort]

On Mon, Mar 6, 2017 at 1:03 AM, David Schmitt <david....@puppet.com> wrote:

On 5 March 2017 at 05:36, David Lutterkort <lut...@puppet.com> wrote:

If anybody is interested, I just put up a PR that adds mruby - there are a couple rough edges in terms of setup/installation, but other than those, things seem to work. I am starting to like this mruby thing more and more ...

The PR also has a small bridge between current libral API and (some approximation) of the t&p proposal from puppet-dev - that bridge should go away when I rework the libral API.

If you're referring to Ral::CLI in https://github.com/puppetlabs/libral/pull/9/files#diff-6a615615c290551b1800bc22d3a4bf34R108 , I couldn't find the code for that. Did you add it to the commit?

It's part of the mruby-libral mrbgem that gets built into mruby; this commit has that: https://github.com/puppetlabs/libral/pull/9/commits/01ba3178be7dda2b8957f9c90038cc181099492d

David