[erlang-questions] [PATCH] Fix R15B02 cross-compilation for TileraMDE-3.0.1.125620

[Motiejus Jakštys]

Applying the patch below let me cross-compile and run R15B02 on Tilera64
using TileraMDE-3.0.1.125620.

In order to prepare a clean patch, I have to know what do the options
-OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off do on the machines that
support it? Also, what is the meaning of -MALLOC_USE_HASH(1)?

Overall, thumbs up to the OTP team for making cross compilation very easy.
:)

---

-OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off
tile-gcc 4.4.3 does accept these options:

cc1: error: unrecognized command line option
"-WOPT:lpre=off:spre=off:epre=off"

MALLOC_USE_HASH is not defined anywhere in the mentioned build
environment.
---
erts/emulator/Makefile.in | 1 -
erts/emulator/sys/unix/sys.c | 6 ------
2 files changed, 7 deletions(-)

diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in
index 985ef72..644ca18 100644
--- a/erts/emulator/Makefile.in
+++ b/erts/emulator/Makefile.in
@@ -633,7 +633,6 @@ endif
ifneq ($(filter tile-%,$(TARGET)),)
$(OBJDIR)/beam_emu.o: beam/beam_emu.c
$(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) \
- -OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off \
$(INCLUDES) -c $< -o $@
else
# Usually the same as the default rule, but certain platforms (e.g. win32) mix
diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c
index c1fa00b..4c168b6 100644
--- a/erts/emulator/sys/unix/sys.c
+++ b/erts/emulator/sys/unix/sys.c
@@ -398,12 +398,6 @@ void sys_tty_reset(int exit_code)
}
}

-#ifdef __tile__
-/* Direct malloc to spread memory around the caches of multiple tiles. */
-#include <malloc.h>
-MALLOC_USE_HASH(1);
-#endif
-
#ifdef USE_THREADS

#ifdef ERTS_THR_HAVE_SIG_FUNCS
--
1.7.9.5

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Björn-Egil Dahlberg]

On 2012-10-10 16:08, Motiejus Jakštys wrote:

Applying the patch below let me cross-compile and run R15B02 on Tilera64
using TileraMDE-3.0.1.125620.

In order to prepare a clean patch, I have to know what do the options
-OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off do on the machines that
support it? Also, what is the meaning of -MALLOC_USE_HASH(1)?

I am quoting from Tilera here:

"'malloc_use_hash' only affects malloc (specifically, it affects the
flags it passes to 'mmap' when it allocates more memory from Linux).
You can also explicitly malloc memory with various caching attributes
using custom mspaces.

The 'hash_default' argument in the .hvconfig file affects 'mmap' calls
that do not specify homing information.  Because malloc *does* specify
homing information, at least when you are pthreaded, 'hash_default' is
ignored and malloc_use_hash is required to get hash-for-home."

This was something included in TileraMDE 2.0 beta (perhaps removed since then).
I found it in TileraMDE-2.1.2/tile64/tile/usr/include/malloc.h though.

I guess it would be safe to remove "-OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off ".
It is an optimization for tile-cc when compiling on target machine.
An earlier comment in Makefile.in seems to have been removed, it said:

+ifneq ($(filter tile-%,$(TARGET)),)
+# Some tile-cc optimizations take pathologically long for beam_emu.c,
+# so disable them.
+$(OBJDIR)/beam_emu.o: beam/beam_emu.c
+	$(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) \
+	  -OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off \
+	  $(INCLUDES) -c $< -o $@
+endif

Patch was supplied by Tilera Corp.

Regards,
Björn-Egil Dahlberg
Erlang/OTP

[Motiejus Jakštys]

On Wed, Oct 10, 2012 at 05:09:10PM +0200, Björn-Egil Dahlberg wrote:

> On 2012-10-10 16:08, Motiejus Jaks(tys wrote:
> >Applying the patch below let me cross-compile and run R15B02 on Tilera64
> >using TileraMDE-3.0.1.125620.
> >
> >In order to prepare a clean patch, I have to know what do the options
> >-OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off do on the machines that
> >support it? Also, what is the meaning of -MALLOC_USE_HASH(1)?
>
> I am quoting from Tilera here:

Is documentation open? Would it be possible to get a link/reference?
Maybe there is any documentation on this feature in "compatibility
notes" or "migration guide" of some kind? I could ask Tilera for this.

> "'malloc_use_hash' only affects malloc (specifically, it affects the
> flags it passes to 'mmap' when it allocates more memory from Linux).
> You can also explicitly malloc memory with various caching attributes
> using custom mspaces.
>
> The 'hash_default' argument in the .hvconfig file affects 'mmap' calls

> that do not specify homing information. Because malloc*does* specify

> homing information, at least when you are pthreaded, 'hash_default' is
> ignored and malloc_use_hash is required to get hash-for-home."
>
>
> This was something included in TileraMDE 2.0 beta (perhaps removed
> since then).
> I found it in TileraMDE-2.1.2/tile64/tile/usr/include/malloc.h though.

All right. So it was there, and was removed.

I think it would make sense to conditionalize it during configuration
phase. But how to test for its existence/absence?

> I guess it would be safe to remove "-OPT:Olimit=0
> -WOPT:lpre=off:spre=off:epre=off ".
> It is an optimization for tile-cc when compiling on target machine.
> An earlier comment in Makefile.in seems to have been removed, it said:
>
> +ifneq ($(filter tile-%,$(TARGET)),)
> +# Some tile-cc optimizations take pathologically long for beam_emu.c,
> +# so disable them.
> +$(OBJDIR)/beam_emu.o: beam/beam_emu.c
> + $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) \
> + -OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off \
> + $(INCLUDES) -c $< -o $@
> +endif

Can you verify that it compiles in "reasonable time" on your system with
these flags turned off?

> Patch was supplied by Tilera Corp.

That is fantastic.

Motiejus

[Björn-Egil Dahlberg]

On 2012-10-11 08:45, Motiejus Jakštys wrote:
> On Wed, Oct 10, 2012 at 05:09:10PM +0200, Björn-Egil Dahlberg wrote:
>> On 2012-10-10 16:08, Motiejus Jaks(tys wrote:
>>> Applying the patch below let me cross-compile and run R15B02 on Tilera64
>>> using TileraMDE-3.0.1.125620.
>>>
>>> In order to prepare a clean patch, I have to know what do the options
>>> -OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off do on the machines that
>>> support it? Also, what is the meaning of -MALLOC_USE_HASH(1)?
>> I am quoting from Tilera here:
> Is documentation open? Would it be possible to get a link/reference?
> Maybe there is any documentation on this feature in "compatibility
> notes" or "migration guide" of some kind? I could ask Tilera for this.

I'm sorry. It's not possible to make that specific information public. I
was quoting from a private
mail discussion I had with people at Tilera in 2008.

>> "'malloc_use_hash' only affects malloc (specifically, it affects the
>> flags it passes to 'mmap' when it allocates more memory from Linux).
>> You can also explicitly malloc memory with various caching attributes
>> using custom mspaces.
>>
>> The 'hash_default' argument in the .hvconfig file affects 'mmap' calls
>> that do not specify homing information. Because malloc*does* specify
>> homing information, at least when you are pthreaded, 'hash_default' is
>> ignored and malloc_use_hash is required to get hash-for-home."
>>
>>
>> This was something included in TileraMDE 2.0 beta (perhaps removed
>> since then).
>> I found it in TileraMDE-2.1.2/tile64/tile/usr/include/malloc.h though.
> All right. So it was there, and was removed.
>
> I think it would make sense to conditionalize it during configuration
> phase. But how to test for its existence/absence?

How about doing this instead?

diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c
index 37dfcb1..176a5af 100644
--- a/erts/emulator/sys/unix/sys.c
+++ b/erts/emulator/sys/unix/sys.c
@@ -408,8 +408,10 @@ void sys_tty_reset(int exit_code)
#ifdef __tile__

/* Direct malloc to spread memory around the caches of multiple tiles. */

#include <malloc.h>
+#if defined(MALLOC_USE_HASH)
MALLOC_USE_HASH(1);
#endif
+#endif

#ifdef USE_THREADS

>> I guess it would be safe to remove "-OPT:Olimit=0
>> -WOPT:lpre=off:spre=off:epre=off ".
>> It is an optimization for tile-cc when compiling on target machine.
>> An earlier comment in Makefile.in seems to have been removed, it said:
>>
>> +ifneq ($(filter tile-%,$(TARGET)),)
>> +# Some tile-cc optimizations take pathologically long for beam_emu.c,
>> +# so disable them.
>> +$(OBJDIR)/beam_emu.o: beam/beam_emu.c
>> + $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) \
>> + -OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off \
>> + $(INCLUDES) -c $< -o $@
>> +endif
> Can you verify that it compiles in "reasonable time" on your system with
> these flags turned off?

Actually, no. =)

This was a problem Tilera had when they compiled in their environment,
i.e. uploading all code to the card and compiling it there (With their
specific cards). Here at OTP we cross-compiled everything and did not
see this problem. Hopefully this was a problem that has been remedied
since then though.

Also, tag - you’re it!

It would be greatly appreciated if you could summarize problems and
solution you have found for building and using Erlang on the Tilera card
or if you find something we can change to make anyones experience just a
bit easier. I don't know exactly where we should put this documentation
but my suggestion would either be in the documentation for cross
compiling or as a specific page on the GitHub Wiki. If the information
arrives, we will surely find a place to put it. =)

// Björn-Egil

[Motiejus Jakštys]

On Thu, Oct 11, 2012 at 12:10:39PM +0200, Björn-Egil Dahlberg wrote:
> On 2012-10-11 08:45, Motiejus Jakštys wrote:
> >On Wed, Oct 10, 2012 at 05:09:10PM +0200, Björn-Egil Dahlberg wrote:
> >>"'malloc_use_hash' only affects malloc (specifically, it affects the
> >>flags it passes to 'mmap' when it allocates more memory from Linux).
> >>You can also explicitly malloc memory with various caching attributes
> >>using custom mspaces.
> >>

> >I think it would make sense to conditionalize it during configuration
> >phase. But how to test for its existence/absence?
> How about doing this instead?
>
> diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c
> index 37dfcb1..176a5af 100644
> --- a/erts/emulator/sys/unix/sys.c
> +++ b/erts/emulator/sys/unix/sys.c
> @@ -408,8 +408,10 @@ void sys_tty_reset(int exit_code)
> #ifdef __tile__
> /* Direct malloc to spread memory around the caches of multiple tiles. */
> #include <malloc.h>
> +#if defined(MALLOC_USE_HASH)
> MALLOC_USE_HASH(1);
> #endif
> +#endif
> #ifdef USE_THREADS

Oh, it was a macro. Of course. :-)

> >>I guess it would be safe to remove "-OPT:Olimit=0
> >>-WOPT:lpre=off:spre=off:epre=off ".
> >>It is an optimization for tile-cc when compiling on target machine.
> >>An earlier comment in Makefile.in seems to have been removed, it said:
> >>
> >>+ifneq ($(filter tile-%,$(TARGET)),)
> >>+# Some tile-cc optimizations take pathologically long for beam_emu.c,
> >>+# so disable them.
> >>+$(OBJDIR)/beam_emu.o: beam/beam_emu.c
> >>+ $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) \
> >>+ -OPT:Olimit=0 -WOPT:lpre=off:spre=off:epre=off \
> >>+ $(INCLUDES) -c $< -o $@
> >>+endif
> >Can you verify that it compiles in "reasonable time" on your system with
> >these flags turned off?

> Also, tag - you’re it!

Did not understand the above sentence.

> It would be greatly appreciated if you could summarize problems and
> solution you have found for building and using Erlang on the Tilera
> card or if you find something we can change to make anyones
> experience just a bit easier. I don't know exactly where we should
> put this documentation but my suggestion would either be in the
> documentation for cross compiling or as a specific page on the
> GitHub Wiki. If the information arrives, we will surely find a place
> to put it. =)

When one is doing
./otp_build autoconf[*]
./otp_build configure --xcomp-conf=FILE
make

Then everything in documentation makes sense and is really clear.

However, if one wants to cross-compile using ./configure[⚗], then you have
to know the cpu-vendor-os triplet. I got it while invoking GCC on the
target machine:

$ tile-monitor --resume --pci --resume --root -- gcc -v
Using built-in specs.
Target: tile-unknown-linux-gnu
...

Is there a better way? Anyway, this or the better way, I think, could be
documented.

If people run into compilation problems, usually it is not Erlang
specific. It's a missing macro, invalid compiler flag, missing function
during linking.. Standard stuff which is common to any C project. I am
afraid it does not make sense to add help to the Erlang building
documentation. Like "try removing stuff that is complaining and see
what happens" (the way I did) is too easy to be misinterpreted, and
whilst if you provide more aid you just repeat general C knowledge.

Sadly, specific questions are asked, without a reply[1].

Anyway, regarding code. Should I "prepare" a new patch with an updated
commit message, or what you have is good enough to be merged?

Regards,
Motiejus

[*]: I am building from git source.
[⚗]: I used ./otp_build only, and this is speculation. Not sure if would work.
[1]: http://erlang.org/pipermail/erlang-questions/2011-October/061727.html

[Björn-Egil Dahlberg]

On 2012-10-11 19:15, Motiejus Jakštys wrote:
> Anyway, regarding code. Should I "prepare" a new patch with an updated
> commit message, or what you have is good enough to be merged?

Either send a patch through via our patch-process, see
https://github.com/erlang/otp/wiki/Submitting-patches

Or I can make the changes with the additional small changes I proposed.

// Björn-Egil

[Motiejus Jakštys]

On Thu, Oct 18, 2012 at 02:24:00PM +0200, Björn-Egil Dahlberg wrote:
> On 2012-10-11 19:15, Motiejus Jakštys wrote:
> >Anyway, regarding code. Should I "prepare" a new patch with an updated
> >commit message, or what you have is good enough to be merged?
> Either send a patch through via our patch-process, see
> https://github.com/erlang/otp/wiki/Submitting-patches

Patch sent, hopefully correctly.

Regards,
Motiejus