[erlang-questions] HiPE compiler

[Matthew Evans]

Hi group,

My understanding is that HiPE takes Erlang byte code beam files as the input to produce modified combined HiPE + byte code beam files.

I'm thinking of using HiPE on a project. Unfortunately the build system is mixed, and I'd rather not go down the cross-compiler route. My question is - is there a way to create "HiPE targets" just from beam files on the target platform? The source will, obviously, not be available on the target platform.

Thanks

Matt

[Kostis Sagonas]

I am not sure I understand your question well, but I'll give it a shot
in answering it anyway.

Unlike BEAM byte code, the native code that the HiPE compiler generates
has a very strong dependence on the specifics of the Erlang runtime
system which is running the code. This means that you cannot native
compile the code using Erlang/OTP R-X and expect it to run on R-Y where
X and Y are different. This holds even for minor revisions of the same
major release (i.e. code compiled for R14B03 cannot run on R14B03.)
Also obviously there is a strong dependency on the target architecture:
i.e. you cannot compile code for an x86 and expect it to run on an ARM.

But you can compile your code natively and run it on another target if
the machine that you compile in and the target machine are running
exactly the same Erlang/OTP release and have the same underlying
architecture. In this case you can use:

erlc +native FILE.erl -o FILE.beam

and transfer these FILE.beam(s) on the target machine. I am pretty sure
this works.

But what you can also do the following: compile the files to BEAM byte
code and transfer these files to the target machine and then native
compile these files on the fly (as part of the start of your
application) by using something like the following:

lists:foreach(fun (F) -> hipe:c(F) end, BEAM_FILES)

This will compile the BEAM files in memory, generate native code for
them and load them in the running system. (Obviously, you can choose to
compile only a subset of your files, perhaps only those that are time
critical.) There is obviously a small start up cost in doing this, but
in most applications this should not be an issue.

Let me know how it goes. We can continue this perhaps offline.

Kostis
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Matthew Evans]

Thanks Kostis,

Calling hipe:c/1 on the beam file was exactly what I wanted. I thought you could only run it on source files.

Matt

> Date: Sun, 29 Jan 2012 22:35:23 +0200
> From: kos...@cs.ntua.gr
> To: erlang-q...@erlang.org
> Subject: Re: [erlang-questions] HiPE compiler

[Daniel Luna]

Just remember to compile your files with debug_info for this to work.
HiPE needs the information in there to do its magic.

/Daniel

[Björn Gustavsson]

On Mon, Jan 30, 2012 at 3:11 AM, Daniel Luna <dan...@lunas.se> wrote:
> Just remember to compile your files with debug_info for this to work.
> HiPE needs the information in there to do its magic.

No, that is not correct. HiPE does not the abstract code in the
BEAM file. It uses the BEAM byte code itself to generate native
code.

By the way, there is an easy way add native code to existing
BEAM files, with no need for the source code:

erlc +native FILE.beam

--
Björn Gustavsson, Erlang/OTP, Ericsson AB

[Kostis Sagonas]

On 01/30/12 04:11, Daniel Luna wrote:
> Just remember to compile your files with debug_info for this to work.
> HiPE needs the information in there to do its magic.

That's NOT the case! (Daniel may be confusing HiPE with dialyzer.) No
special options are needed for this to work.

[József Bérces]

Hi,

It seems that even after compiling erlang with hipe support, the system modules do not have the native code. For example, beam_lib:info does not show any HX86 section for lists.

Would the system benefit from compiling all files with native code? If yes, is there an easy way to do that? Maybe I missed some switches during compilation?

Thanks,
Jozsef

[Paul Fisher]

--enable-native-libs

--
paul

[Kostis Sagonas]

On 02/02/12 05:54, József Bérces wrote:
> Hi,
>
> It seems that even after compiling erlang with hipe support, the system modules do not have the native code. For example, beam_lib:info does not show any HX86 section for lists.
>
> Would the system benefit from compiling all files with native code? If yes, is there an easy way to do that? Maybe I missed some switches during compilation?

As somebody else already mentioned, you can use the configure option:
--enable-native-libs.

However, this currently does not compile all files to native code. It
only enables native code compilation of files in kernel, stdlib, hipe,
dialyzer, typer and syntax_tools. There is no fundamental reason for
this: it is just laziness from my part that I've never bothered adding
the following lines to the Makefiles of the remaining applications:

ifeq ($(NATIVE_LIBS_ENABLED),yes)
ERL_COMPILE_FLAGS += +native
endif

Ideally, (something like) the above should be added in the `master'
Makefile instead of on each and every individual Makefile. Perhaps
somebody at OTP who knows the Makefile structure better than I do can do
this change.

Kostis