Re: [ansible-project] binary module seems does not work

[Michael DeHaan]

I don't think there's going to be that many needs where the system will require a native binary module for speed reasons, though I would support patches that detected the file was binary and attempted

to /not/ process it or transfer it differently, I guess, if they were small and simple enough.  

It's just not something many people are going to use.

As for why we modify modules, there are definitely some things we do with modules to replace shebang lines on different systems (suppose one uses one python, another another, and one isn't in the user's path because we don't trust the user) -- this is the ansible_python_interpreter mechanism.   What we do with our own python modules to embed arguments and optimize the way data is transferred in another story, this would be about the 'non-newstyle-module' code path.

--Michael

On Mon, Apr 22, 2013 at 11:19 AM, Shirou w <shiro...@gmail.com> wrote:

Hi, all

I made a simple time module by using golang (http://golang.org/).

It can run with "test-module" but when I run with ansible command, did not work.

% ansible -vvv all -m time -M . --ask-pass

SSH password:

<127.0.0.1> ESTABLISH CONNECTION FOR USER: sample on PORT 22 TO 127.0.0.1

<127.0.0.1> EXEC /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-1366642710.56-42277018789729 && chmod a+rx $HOME/.ansible/tmp/ansible-1366642710.56-42277018789729 && echo $HOME/.ansible/tmp/ansible-1366642710.56-42277018789729'

<127.0.0.1> REMOTE_MODULE time.go 

127.0.0.1 | FAILED => failure encoding into utf-8

This is caused by _copy_module in the runner/__init__.py. 

I am new to ansible but I think the _copy_module assumes only string module.

Is this true or wrong? if true, ansible will not support binary module?

BTW, golang makes static linked binary which can just copy to the remote and run. 

It may become good solution to writing module which requires speed.

(ah, the file size is big just a little...)

Thank you.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[shirou]

Go-lang compiler makes a static-linked binary that includes all of the
required libraries. This means a module becomes more independent from
target hosts environment. No other files (such as libc.so) are
required to execute that module.
That's why I want to use go-lang for writing modules. speed is second ;)

Anyway, thank you for accepting my concern.

I like the design principles of ansible-project and I keep on using ansible.

Thank you
--
WAKAYAMA Shirou


2013/4/23 Michael DeHaan <mic...@ansibleworks.com>:

[Michael DeHaan]

Tell you what, do you have a sample source for a go module that produces JSON output?

Just a "Hello World" is ok -- I just don't know enough go to write one.

I could then see what it would take to transfer it unaltered.

[shirou]

Here is the sample time module source.

https://gist.github.com/shirou/5444078

This is almost same as http://ansible.cc/docs/moduledev.html#reading-input.

This is a sample output using test-module.
(note: time format is different from other language.)

% ./test-module -m ./main -a "f=2006" # with argument
***********************************
RAW OUTPUT
{"time":"2013"}

%./test-module -m ./main # without argument
***********************************
RAW OUTPUT
{"time":"2013-04-22 23:37:13"}

Since go-lang has JSON library in core library, it is easy to produce
JSON output.

--
WAKAYAMA Shirou


2013/4/23 Michael DeHaan <michael...@gmail.com>:

[Kelsey Hightower]

On Tuesday, April 23, 2013 7:44:26 AM UTC-7, Shirou w wrote:

Here is the sample time module source.

https://gist.github.com/shirou/5444078

This is almost same as http://ansible.cc/docs/moduledev.html#reading-input.

This is a sample output using test-module.
(note: time format is different from other language.)

   % ./test-module -m ./main -a "f=2006"  # with argument
   ***********************************
   RAW OUTPUT
   {"time":"2013"}

   %./test-module -m ./main  # without argument
   ***********************************
   RAW OUTPUT
   {"time":"2013-04-22 23:37:13"}

Since go-lang has JSON library in core library, it is easy to produce
JSON output.

--
WAKAYAMA Shirou


2013/4/23 Michael DeHaan <michael...@gmail.com>:
> Tell you what, do you have a sample source for a go module that produces
> JSON output?
>
> Just a "Hello World" is ok -- I just don't know enough go to write one.
>
> I could then see what it would take to transfer it unaltered.

Did this ever get resolved? Michael, did you ever get this to work?

[Michael DeHaan]

This isn't something I've spent time on, but it seems like it would be a pretty basic patch that I would consider.

Still interested in it for Go purposes?   Curious about the use case knowing who you work for :)

--Michael

[shirou]

Hi Michael and Kelsey,

I am writing modules by python because I am familiar than golang (currently!)

However, I think this is not only golang specific issue.
On the ansible website[1], "any language is possible. " it includes C++.
Some people may want to write modules by C++ with existing libraries.


[1] http://www.ansibleworks.com/docs/moduledev.html

Regards,
WAKAYAMA Shirou


2013/8/21 Michael DeHaan <mic...@ansibleworks.com>:

[Michael DeHaan]

Yes, we're aware!  

It should be possible to detect the module is binary and not look for the shebang in it, and otherwise treat it like a non-Python module in another language.

One thing that we /may/ wish to do is automatically send binary modules JSON, rather than the key=value legacy stuff it currently does by default (see "WANTS_JSON" in the code for the hack we did for the Ruby modules).

(I should aspect Go should have a decent JSON library available, right?)

Then the module would just need to look at argv[1] and read in that file as JSON and should be fine.

I would suspect there's not a HUGE advantage to doing this, as while the module is now binary, the arguments file must be transferred seperately -- I'd almost bet the Python method is still faster and you can still load C libraries from there.

[Olivier Wulveryck]

Hello,

I'm facing the exact same problem today.
I've developed a module in go (actually mainly because I know this language, and also because I've a bunch of library I'm using to connect to a specific backend).
Anyway, the module is running ok with the test-module utility (as described in the ansible documentation), but I'm totally unable to make it run within a playbook

The error is "module (mymodule) is missing interpreter line".

I'm aware that this thread is more than 2 years old, but I cannot find any recent information on google and the documentation keeps on mentioning that it is possible to develop a module in *any* language.

Do you have any workaround to propose to use a binary module? Were you able to achieve your goal?

Regards,

Olivier

[Matt Martz]

There is a pull request at https://github.com/ansible/ansible/pull/13771 to add support for binary modules.  The plan is to merge it soon after 2.1 is released.  Support for binary modules will be included in the 2.2 release.

To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/852f7814-35e5-4672-8a03-b8ed84d6bf05%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[Brian Coca]

no support in ansible yet, but might have soon thanks to this https://github.com/ansible/ansible/pull/13771

----------

Brian Coca