Using gcc from Hashstack to compile Hashstack (bootstrapping)

[Ondřej Čertík]

Hi,

On RHEL6, some packages cannot be compiled with the default compilers
(gcc 4.4). So I first created a profile that compiles gcc and few
other packages (Python, Perl, ...):

https://gist.github.com/certik/c4a5eb1ce96ddca52767#file-hd_base-yaml

using the default compilers (gcc 4.4). Then I load it using PROLOGUE,
there are two ways to do it, both seem to work:

a) Using LD_LIBRARY_FLAGS

https://gist.github.com/certik/c4a5eb1ce96ddca52767#file-test-yaml

parameters:
PROLOGUE: |
export HASHSTACK_BASE=/home/certik/repos/hashstack/hd_base; export
PATH="$HASHSTACK_BASE/bin:$PATH"; export
LD_LIBRARY_PATH="$HASHSTACK_BASE/lib"; export CC=gcc CXX=g++
FC=gfortran


b) Using LDFLAGS (requires https://github.com/hashdist/hashstack/pull/643)

https://gist.github.com/certik/c4a5eb1ce96ddca52767#file-test2-yaml

parameters:
PROLOGUE: |
export HASHSTACK_BASE=/home/certik/repos/hashstack/hd_base; export
PATH="$HASHSTACK_BASE/bin:$PATH"; export CC=gcc CXX=g++ FC=gfortran
LDFLAGS="-Wl,-rpath=$HASHSTACK_BASE/lib"



The reason why you need a) or b) is that all executables created with
g++ must link against the standard C++ library from Hashstack, not the
systemwide, otherwise they won't run (the systemwide is too old).

The huge advantage of b) is that no LD_LIBRARY_FLAGS settings are
required, but it is a bit fragile, first of all, #643 needs to be
merged for it to work, and second, maybe some packages will fail
(though quite a few work just fine, including e.g. Trilinos).

The advantage of a) is that it is more robust, the disadvantage is
that now the user must set LD_LIBRARY_PATH by hand, otherwise programs
will not run from the profile. Also, by setting it on RHEL6, some
systemwide programs (e.g. eog, evince, ...) stop working (segfault).
So from this perspective, b) is much better.

Let me know what you think and which way we should officially bless.

Ondrej

[Aron Ahmadia]

I definitely prefer b. to a., but this doesn't look right to me.  If your hashddist-built compiler is correctly configured/built, it should be linking against its own library, not the system one, by default.  Sorry, I don't have time to look into this more.

-A

--
You received this message because you are subscribed to the Google Groups "hashdist" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hashdist+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[François Bissey]

Realised I only sent this to Ondrej and not the list:

I have to agree that LD_LIBRARY_PATH is a kludge that should be avoided if we can. I'll note that both of your solution are linux centric in one
way or another.

For linux there is another more elegant solution. Patch gcc
so that its spec file (gcc -dumpspecs) includes "-Wl,--rpath=...".
Even package that don't respect LDFLAGS will get it. The only
packages for which that will fail are those using ld directly.
Those will also fail using option (b). 

One addition: if you really, really want everything to have the correct rpath 

you may want to wrap ld to do it.

Francois

[Ondřej Čertík]

On Tue, Feb 17, 2015 at 4:47 PM, François Bissey
<francoi...@canterbury.ac.nz> wrote:
> Realised I only sent this to Ondrej and not the list:
>
> I have to agree that LD_LIBRARY_PATH is a kludge that should be avoided if
> we can. I'll note that both of your solution are linux centric in one
> way or another.
>
> For linux there is another more elegant solution. Patch gcc
> so that its spec file (gcc -dumpspecs) includes "-Wl,--rpath=...".
> Even package that don't respect LDFLAGS will get it. The only
> packages for which that will fail are those using ld directly.
> Those will also fail using option (b).

That would be an ideal solution. I didn't know it's possible. That
would fix everything.

Do you have experience with this --- i.e. where should this be put?

Ondrej

[François Bissey]

On 02/18/15 13:41, Ondřej Čertík wrote:
> Do you have experience with this --- i.e. where should this be put?

Yes I do. It is all under gcc/config/ where you have files in various
architecture directory here is an example from the cluster I manage
(there is some extra stuff because I have also use an external glibc)

diff -Naur gcc-4.9.1/gcc/config//rs6000/linux64.h
gcc-4.9.1_at/gcc/config//rs6000/linux64.h
--- gcc-4.9.1/gcc/config//rs6000/linux64.h 2014-01-03
11:23:26.000000000 +1300
+++ gcc-4.9.1_at/gcc/config//rs6000/linux64.h 2014-10-09
13:13:58.291446000 +1300
@@ -367,11 +367,11 @@
#undef LINK_OS_DEFAULT_SPEC
#define LINK_OS_DEFAULT_SPEC "%(link_os_linux)"

-#define GLIBC_DYNAMIC_LINKER32 "/lib/ld.so.1"
+#define GLIBC_DYNAMIC_LINKER32 "/opt/at7.0/lib/ld.so.1"
#ifdef LINUX64_DEFAULT_ABI_ELFv2
-#define GLIBC_DYNAMIC_LINKER64
"%{mabi=elfv1:/lib64/ld64.so.1;:/lib64/ld64.so.2}"
+#define GLIBC_DYNAMIC_LINKER64
"/opt/at7.0%{mabi=elfv1:/lib64/ld64.so.1;:/lib64/ld64.so.2}"
#else
-#define GLIBC_DYNAMIC_LINKER64
"%{mabi=elfv2:/lib64/ld64.so.2;:/lib64/ld64.so.1}"
+#define GLIBC_DYNAMIC_LINKER64
"/opt/at7.0%{mabi=elfv2:/lib64/ld64.so.2;:/lib64/ld64.so.1}"
#endif
#define UCLIBC_DYNAMIC_LINKER32 "/lib/ld-uClibc.so.0"
#define UCLIBC_DYNAMIC_LINKER64 "/lib/ld64-uClibc.so.0"
@@ -406,13 +406,13 @@
" -m elf64ppc")
#endif

-#define LINK_OS_LINUX_SPEC32 LINK_OS_LINUX_EMUL32 " %{!shared: %{!static: \
+#define LINK_OS_LINUX_SPEC32 LINK_OS_LINUX_EMUL32 " %{!static: %{!shared: \
%{rdynamic:-export-dynamic} \
- -dynamic-linker " GNU_USER_DYNAMIC_LINKER32 "}}"
+ -dynamic-linker " GNU_USER_DYNAMIC_LINKER32 "}} --enable-new-dtags
-rpath /usr/local/pkg/gcc/4.9/lib64:/opt/at7.0/lib"

-#define LINK_OS_LINUX_SPEC64 LINK_OS_LINUX_EMUL64 " %{!shared: %{!static: \
+#define LINK_OS_LINUX_SPEC64 LINK_OS_LINUX_EMUL64 " %{!static: %{!shared: \
%{rdynamic:-export-dynamic} \
- -dynamic-linker " GNU_USER_DYNAMIC_LINKER64 "}}"
+ -dynamic-linker " GNU_USER_DYNAMIC_LINKER64 "}} --enable-new-dtags
-rpath /usr/local/pkg/gcc/4.9/lib64:/opt/at7.0/lib64"

#undef TOC_SECTION_ASM_OP
#define TOC_SECTION_ASM_OP \
diff -Naur gcc-4.9.1/gcc/config//rs6000/sysv4.h
gcc-4.9.1_at/gcc/config//rs6000/sysv4.h
--- gcc-4.9.1/gcc/config//rs6000/sysv4.h 2014-02-11
03:46:24.000000000 +1300
+++ gcc-4.9.1_at/gcc/config//rs6000/sysv4.h 2014-10-09
13:14:28.170615000 +1300
@@ -761,7 +761,7 @@

#define LINK_START_LINUX_SPEC ""

-#define GLIBC_DYNAMIC_LINKER "/lib/ld.so.1"
+#define GLIBC_DYNAMIC_LINKER "/opt/at7.0/lib/ld.so.1"
#define UCLIBC_DYNAMIC_LINKER "/lib/ld-uClibc.so.0"
#if DEFAULT_LIBC == LIBC_UCLIBC
#define CHOOSE_DYNAMIC_LINKER(G, U) "%{mglibc:" G ";:" U "}"
@@ -773,9 +773,9 @@
#define GNU_USER_DYNAMIC_LINKER \
CHOOSE_DYNAMIC_LINKER (GLIBC_DYNAMIC_LINKER, UCLIBC_DYNAMIC_LINKER)

-#define LINK_OS_LINUX_SPEC "-m elf32ppclinux %{!shared: %{!static: \
+#define LINK_OS_LINUX_SPEC "-m elf32ppclinux %{!static: %{!shared: \
%{rdynamic:-export-dynamic} \
- -dynamic-linker " GNU_USER_DYNAMIC_LINKER "}}"
+ -dynamic-linker " GNU_USER_DYNAMIC_LINKER "}} --enable-new-dtags
-rpath /usr/local/pkg/gcc/4.9/lib:/opt/at7.0/lib"

#if defined(HAVE_LD_EH_FRAME_HDR)
# define LINK_EH_SPEC "%{!static:--eh-frame-hdr} "
@@ -804,7 +804,7 @@
"

#define LINK_OS_NETBSD_SPEC "\
-%{!shared: %{!static: \
+%{!static: %{!shared: \
%{rdynamic:-export-dynamic} \
-dynamic-linker /usr/libexec/ld.elf_so}}"


You would be only concerned by the bits added that have rpath in it.
/opt/at7.0/lib{64} is where my glibc is located. This is the rs6000
folder because I am dealing with ppc64 on x86(_64) you'll want
gcc/config/i386. If you want to generalize you have to correct all
file called linux.h, linux64.h and sysv4.h in the subdirectories of
gcc/config.

I don't know how to apply this to darwin, furthermore not all version
of OS X have an ld that accept rpath.

Francois

[Ondřej Čertík]

I see, thanks for the pointer. How about using the --with-specs option
during configure? From https://gcc.gnu.org/install/configure.html

--with-specs=specs Specify additional command line driver SPECS. This
can be useful if you need to turn on a non-standard feature by default
without modifying the compiler's source code, for instance
--with-specs=%{!fcommon:%{!fno-common:-fno-common}}. See “Spec Files”
in the main manual


I.e. something like (from http://xenogenesi.altervista.org/gcc-file-specs/):

--with-specs="%{shared:-Wl,-rpath
-Wl,$(DESTDIR)/lib}%{!shared:-Wl,-rpath -Wl,$(DESTDIR)/lib}"

It looks like this might be it. Then DESTDIR must be exposed into the
profile, so that the user can set it, and that's it.

Ondrej

[François Bissey]

On 02/18/15 14:07, Ondřej Čertík wrote:
> I see, thanks for the pointer. How about using the --with-specs option

> during configure? Fromhttps://gcc.gnu.org/install/configure.html

>
> --with-specs=specs Specify additional command line driver SPECS. This
> can be useful if you need to turn on a non-standard feature by default
> without modifying the compiler's source code, for instance
> --with-specs=%{!fcommon:%{!fno-common:-fno-common}}. See “Spec Files”
> in the main manual
>
>

> I.e. something like (fromhttp://xenogenesi.altervista.org/gcc-file-specs/):

>
> --with-specs="%{shared:-Wl,-rpath
> -Wl,$(DESTDIR)/lib}%{!shared:-Wl,-rpath -Wl,$(DESTDIR)/lib}"
>
> It looks like this might be it. Then DESTDIR must be exposed into the
> profile, so that the user can set it, and that's it.

Interesting, I didn't know about that option. I note that it says it
isn't shown when you -dumpspecs which may make life difficult when
debugging.

specs used to be stored in a text file that you could manually edit.
But now it is all gone and compiled in. There is however an inheritance
from that time. You can tell gcc to use a specific specs file with
gcc -specs=<file>
Unfortunately there is no way to set that behavior thanks to an
environment variable (IBM XL compiler behavior can be altered that way).

Francois

[Dag Sverre Seljebotn]

On 02/18/2015 12:47 AM, François Bissey wrote:
> Realised I only sent this to Ondrej and not the list:
>
> I have to agree that LD_LIBRARY_PATH is a kludge that should be avoided
> if we can. I'll note that both of your solution are linux centric in one
> way or another.
>
> For linux there is another more elegant solution. Patch gcc
> so that its spec file (gcc -dumpspecs) includes "-Wl,--rpath=...".
> Even package that don't respect LDFLAGS will get it. The only
> packages for which that will fail are those using ld directly.
> Those will also fail using option (b).
>
> One addition: if you really, really want everything to have the correct
> rpath
> you may want to wrap ld to do it.

I think this one sounds very interesting. Could we simply ship an ld
wrapper that injects --rpath=.., and then not have to worry about RPATH
for any of our build scripts ever again?

If you use system gcc, will it use whichever ld is found in PATH?

Dag Sverre

[Francois Bissey]

Normally yes, it will use ld found in the PATH. It is possible to alter this behaviour
when you configure gcc with:
--with-ld=pathname
see https://gcc.gnu.org/install/configure.html in practice I don’t know of a distro that
does that. It is useful for cross-compiling or doing specific experiments.
You can detect with gcc -v.

At runtime you can change where gcc will look for ld (and as and ar) with -B/some/path.

So I’d say wrapping ld is fairly safe.

François

[Dag Sverre Seljebotn]

And in your opinion, would wrapping ld to always add
--rpath="${HASHDIST_BUILD_RPATH}" to arguments if HASHDIST_BUILD_RPATH
is set be a sane thing to do?

Dag Sverre

>
> François
>

[Dag Sverre Seljebotn]

On 02/18/2015 11:18 AM, Francois Bissey wrote:

> Yes. That is my opinion but I am willing to hear a counter argument.
> But I’ll go further. If you plan to ship libraries that could be competing
> with system libraries you don’t ever want this variable to be empty.
> Unless, of course, you want to always use LD_LIBRARY_PATH
> or it equivalent on your system.
> The other methods with LDFLAGS and specs can have points of failure
> in some rare case. And the very last method not discussed so far, fiddling
> with ld.so.conf, is just totally insane.
>



I love the idea.

So

1) We always run with a wrapped ld in PATH (hashdist_compiler_wrapper
package in build deps?)

2) HASHDIST_BUILD_RPATH is set to ":".join(rpath_of_dependencies)

3) Optionally, run the patchelf tool to strip down RPATH to minimum as
post-processing step


No messing with -Wl,-rpath or similar in any build scripts.

Dag Sverre

[Aron Ahmadia]

On Wed, Feb 18, 2015 at 5:28 AM, Dag Sverre Seljebotn <d.s.se...@astro.uio.no> wrote:

1) We always run with a wrapped ld in PATH (hashdist_compiler_wrapper package in build deps?)

2) HASHDIST_BUILD_RPATH is set to ":".join(rpath_of_dependencies)

3) Optionally, run the patchelf tool to strip down RPATH to minimum as post-processing step

This is a bit optimistic, unfortunately.  There are a number of situations where we have access to environment flags but we're not in a position to bootstrap our own compiler or binary utilities, such as on proprietary HPC systems.  I think this discussion (and bootstrapped compilers) are a very useful thing, but we need to remain cognizant of situations where this won't be feasible.

A

[Ondřej Čertík]

I am interested in getting the gcc in Hashdist working extremely well
and I think this is the last missing piece, and we all agree on that.

However, Dag is also proposing (and that's orthogonal to the above) to
wrap the system ld, so I think that might be possible on HPC systems
as well. But I also feel that our approach with CPPFLAGS and LDFLAGS
is somehow cleaner, as you don't need to modify the toolchain.
Remember, that you need to modify CPPFLAGS anyway, for the include
files, so you can just as well modify LDFLAGS in the same manner.

Ondrej

[Aron Ahmadia]

On Wed, Feb 18, 2015 at 10:54 AM, Ondřej Čertík <ondrej...@gmail.com> wrote:

On Wed, Feb 18, 2015 at 8:25 AM, Aron Ahmadia <ar...@ahmadia.net> wrote:
>
> On Wed, Feb 18, 2015 at 5:28 AM, Dag Sverre Seljebotn
> <d.s.se...@astro.uio.no> wrote:
>>
>>
>> 1) We always run with a wrapped ld in PATH (hashdist_compiler_wrapper
>> package in build deps?)
>>
>> 2) HASHDIST_BUILD_RPATH is set to ":".join(rpath_of_dependencies)
>>
>> 3) Optionally, run the patchelf tool to strip down RPATH to minimum as
>> post-processing step
>
>
> This is a bit optimistic, unfortunately.  There are a number of situations
> where we have access to environment flags but we're not in a position to
> bootstrap our own compiler or binary utilities, such as on proprietary HPC
> systems.  I think this discussion (and bootstrapped compilers) are a very
> useful thing, but we need to remain cognizant of situations where this won't
> be feasible.

I am interested in getting the gcc in Hashdist working extremely well
and I think this is the last missing piece, and we all agree on that.

Yes, better gcc support is something we all can agree on.  I think a complete compiler bootstrapping system also needs to happen.  I discussed this with Volker when I was in Oxford, but I haven't had time to write it up, much less implement it.

A

[Ondřej Čertík]

Ok, this implements it: https://github.com/hashdist/hashstack/pull/645

It seems to work!

Ondrej

[François Bissey]

On 02/19/15 04:25, Aron Ahmadia wrote:
>
> On Wed, Feb 18, 2015 at 5:28 AM, Dag Sverre Seljebotn
> <d.s.se...@astro.uio.no <mailto:d.s.se...@astro.uio.no>> wrote:
>
>
> 1) We always run with a wrapped ld in PATH
> (hashdist_compiler_wrapper package in build deps?)
>

> 2) HASHDIST_BUILD_RPATH is set to ":".join(rpath_of___dependencies)

>
> 3) Optionally, run the patchelf tool to strip down RPATH to minimum
> as post-processing step
>
>
> This is a bit optimistic, unfortunately. There are a number of
> situations where we have access to environment flags but we're not in a
> position to bootstrap our own compiler or binary utilities, such as on
> proprietary HPC systems. I think this discussion (and bootstrapped
> compilers) are a very useful thing, but we need to remain cognizant of
> situations where this won't be feasible.
>
> A

Just for my edification which proprietary HPC systems do you have in
mind? I have to deal with AIX and we would have to wrap ld differently
because it doesn't recognize -rpath. -blibpath would be the flag to use
and unlike -rpath in gnu ld it is not cumulative, only the last
-blibpath argument will be used.

Also earlier OS X ld don't support -rpath. I think it was added in 10.8.

Francois

[Aron Ahmadia]

On Wed, Feb 18, 2015 at 5:33 PM, François Bissey <francoi...@canterbury.ac.nz> wrote:

Just for my edification which proprietary HPC systems do you have in mind? I have to deal with AIX and we would have to wrap ld differently because it doesn't recognize -rpath. -blibpath would be the flag to use
and unlike -rpath in gnu ld it is not cumulative, only the last -blibpath argument will be used.

At minimum, we support Cray HPC because those are publicly available to us at US DoD.  We've also tried to support recent OS X (10.9/10.10) and some variant of Windows via Cygwin.

A

[Ondřej Čertík]

Just an update, that that PR had some issues, but this follow up PR
fixes everything (it is now merged):

https://github.com/hashdist/hashstack/pull/646

Here are my updated scripts for bootstrapping:

https://gist.github.com/certik/db990b8260e77830fa48

The test.yaml tests 109 packages and they all build with the compiled
gcc, and as you can see, all that is needed is to put the new gcc into
PATH in PROLOGUE, and export CC, CXX and FC (some build systems like
CMake need those, otherwise they use the systemwide compiler even if
the "gcc" in PATH is a different one). Here is the full list of
packages that I tested:

$ hit status test.yaml
matplotlib/25spx2kh3a7t [OK]
pyzmq/vyilil3bdrqs [OK]
expat/ssfdx4wswilf [OK]
requests/2vxfr6uzbwbp [OK]
sphinx/zdx4gb7hcrlx [OK]
libunwind/yprlxgcoczus [OK]
pyliblzma/rudks2ois576 [OK]
jinja2/dfutvluqu57o [OK]
ws4py/zmf5frkocgkx [OK]
pygments/jnwt7w7jfq3k [OK]
xz/hw64kubfvpya [OK]
breathe/scax6mjxu6iz [OK]
piranha/xoaq6uxa6qu7 [OK]
csympy/hrvvx3upvego [OK]
gprof2dot/c7eifjxwrwji [OK]
zmq/r5tewxnk2gn4 [OK]
gettext/anj7fb7jsyj6 [OK]
form/feaitassv5lq [OK]
gcc/hq77sgfl6yyk [OK]
cloog/wfdp5tmcwe45 [OK]
mpqc/lator67h7sya [OK]
pytest/bccb7q6izrgw [OK]
png/j54hbp4afvye [OK]
six/m6jnfkwljxsa [OK]
subversion/ut7zhfrr5cw4 [OK]
openblas/2wc6yoy6i5m5 [OK]
gmp/bznzihiibbiz [OK]
pyparsing/hsyejvfpyg5h [OK]
curl/5rcqpplmsfx3 [OK]
cython/dezdtu4xapvb [OK]
freetype/axzacofcvaye [OK]
serf/ee5zjmajl5xi [OK]
coreutils/7zc5tyyt5g2g [OK]
selenium/w2vwewzonbgk [OK]
setuptools/wrfxti7a3aen [OK]
flint/aoogf5psvgdg [OK]
scons/mfbvqcdnbgde [OK]
cln/ac7u7rpiqkr4 [OK]
sympy/46wpojg2ddw3 [OK]
netcdf4f/2k5hn4ycdvb7 [OK]
isl/vmr2lnwfmd7r [OK]
gromacs/d6m5l7mm5eib [OK]
libstdcxx/ielcsuwhxlvg [OK]
mercurial/bvjwxcpmx35d [OK]
apr/sovopxddpgja [OK]
apr-util/n2lnc7igmx3h [OK]
libidn/jynat673pzdv [OK]
m4/v33ckvkruuwa [OK]
mpfr/77vtm5z5c44m [OK]
mpc/hl46ohqieheg [OK]
scipy/mkx7ms33msob [OK]
szip/ogvveqfe3n5j [OK]
libiconv/d4vfhxczgn6m [OK]
diffutils/4iyv5l7k7abu [OK]
yajl/kuz3eqaq25l2 [OK]
docutils/44midoqm46so [OK]
hdf5/wxqezcg6edr2 [OK]
fftw/bqot5shsghot [OK]
ginac/eoehzkrovwjk [OK]
pandas/qd4hlwdbx6k5 [OK]
pytz/53ombezg47gz [OK]
autoconf/t7wgqipufbc6 [OK]
scikit-learn/v5kyvreznlkk [OK]
python-dateutil/xflxcwkkhqvw [OK]
doxygen/ez4hfkkuv6je [OK]
dftatom/o6cnigjp44a7 [OK]
mpmath/ofbjq6n6tebl [OK]
gawk/r24uv5u5j7jg [OK]
nose/rdi5kgaqwrcd [OK]
arb/nymb3ugxgnsl [OK]
pkg-config/7ukxwyldfq6o [OK]
py/sa477eum3vhu [OK]
gettext-bootstrap/gqpnoylglhm3 [OK]
bison/vdilqddc7zkq [OK]
git/mpmt6bc77xps [OK]
gmpy/ddf2rbgfsebl [OK]
automake/elwjizcnljke [OK]
opium/5xrrgtfwuvyf [OK]
tornado/u25a5o7lmkux [OK]
libiconv-bootstrap/klskks3wayyk [OK]
boost/ri26lejezqqu [OK]
patchelf/qqzlbpi7rek7 [OK]
ipython/ebp3tzcopww5 [OK]
scalapack/xfof7j7o3woc [OK]
parmetis/bkdz3lujzqa2 [OK]
gdbm/sli73xv2blrl [OK]
blopex/yu26q6e3iua7 [OK]
blas/46xg6m7tc2sx [OK]
ncurses/nbflhmpnwfh3 [OK]
trilinos/ostvmowsmjgj [OK]
bzip2/jkrxj5nqe34j [OK]
sqlite/q7ioaqjweyy7 [OK]
netcdf4/cijuvziwezdu [OK]
launcher/q3vfxub5kd4x [OK]
zlib/ccismna277ol [OK]
pcre/ah2nkoy47qts [OK]
numpy/25sx7ebwe527 [OK]
openssl/hydesfnrfag4 [OK]
readline/bboihbkzl5tr [OK]
perl/f23d5efumism [OK]
suitesparse/o2f7ezyetviz [OK]
python/xdzveakz2e5j [OK]
berkeleydb-5/xmgjsmvkjlpn [OK]
cmake/eyyhg7azn7ys [OK]
mumps/3v7nvpionxmy [OK]
lapack/h57fnup6t7ej [OK]
mpi/s3onu7dmekim [OK]
bddc/jp3r6ivnt636 [OK]
swig/crdcjpd4c73v [OK]


Ondrej

[Jimmy Tang]

Hi Ondrej

(top posting here). I haven't been following this too closely but +1, it looks like binary packages and some mechanism for easily loading up environments for end users are the last major missing things to make life easier for most users.

I would almost suggest that a new gcc profile which extends hashstack be setup as an example.

Jimmy

[Ondřej Čertík]

On Thu, Feb 19, 2015 at 2:48 AM, Jimmy Tang <jcf...@gmail.com> wrote:
> Hi Ondrej
>
> (top posting here). I haven't been following this too closely but +1, it
> looks like binary packages and some mechanism for easily loading up
> environments for end users are the last major missing things to make life
> easier for most users.

Yes, it is really high priority, but I didn't have time to work on it
yet. I think the environment loading and managing is even higher
priority than binary packages. Without it, Hashdist/Hashstack is
extremely hard to use.

Btw, I still found a problem with the new gcc setup:

https://github.com/hashdist/hashstack/issues/649

I am not sure what the cause is yet, so it needs to be investigated.

>
> I would almost suggest that a new gcc profile which extends hashstack be
> setup as an example.

Yes, I'll do it.

Ondrej

[Ondřej Čertík]

On Thu, Feb 19, 2015 at 10:30 AM, Ondřej Čertík <ondrej...@gmail.com> wrote:
> On Thu, Feb 19, 2015 at 2:48 AM, Jimmy Tang <jcf...@gmail.com> wrote:
>> Hi Ondrej
>>
>> (top posting here). I haven't been following this too closely but +1, it
>> looks like binary packages and some mechanism for easily loading up
>> environments for end users are the last major missing things to make life
>> easier for most users.
>
> Yes, it is really high priority, but I didn't have time to work on it
> yet. I think the environment loading and managing is even higher
> priority than binary packages. Without it, Hashdist/Hashstack is
> extremely hard to use.
>
> Btw, I still found a problem with the new gcc setup:
>
> https://github.com/hashdist/hashstack/issues/649
>
> I am not sure what the cause is yet, so it needs to be investigated.

We figured it out, it got fixed by:

https://github.com/hashdist/hashstack/pull/656

Big thanks goes to François, he figured out the initial as well as the
final patch.

Essentially, I don't think we need to fiddle with LD_LIBRARY_PATH at
all, for any application of Hashstack.
Previously, I had to modify it for gcc, but not anymore. Very exciting
and clean.

I've updated https://gist.github.com/certik/db990b8260e77830fa48, with
just a small change (see
https://gist.github.com/certik/db990b8260e77830fa48/revisions),
essentially the PROLOGUE that I now use is (thanks to Aron):

parameters:
PROLOGUE: |
export PATH="/local/certik/bld/profile/bto56eplrk4v/bin:$PATH";
export CC=gcc CXX=g++ FC=gfortran

where I depend on the exact profile hash, easily obtainable via:

certik@redhawk:~/repos/hashstack$ ll hd_base
lrwxrwxrwx 1 certik certik 38 Feb 20 10:36 hd_base ->
/local/certik/bld/profile/bto56eplrk4v/

So that when I updated the gcc via additional patch, the profile hash
changes, and thus my main profile is still using the old gcc and no
rebuilds are done,
and if I modify the PROLOGUE to use the newer gcc, then it triggers
rebuild of all packages. Which is exactly the ideal behavior.

Finally, this is my .bashrc now:

export HASHSTACK=$HOME/repos/hashstack/hd_base
export PATH="$HASHSTACK/bin:$PATH"
export MANPATH="$HASHSTACK/man:$MANPATH"
export CC=gcc CXX=g++ FC=gfortran

Which seamlessly upgrades my main compiler from the systemwide gcc 4.4
to Hashstack's 4.9.2, and so far it looks like it works perfectly out
of the box.
It needs further testing obviously, to see if I discover any further problems.

Jimmy, if this is going to work well, then the loading of profiles can
essentially really just execute what I put into my .bashrc. The only
questionable line is "export CC=gcc CXX=g++ FC=gfortran", which is
needed for cmake to use the compilers from PATH, otherwise it uses
/usr/bin/gcc, which we do not want.

Ondrej

[Francois Bissey]

Beware that the current stuff only applies to linux on intel/amd hardware.
I can add linux ppc easily and there is a patch to test things on recent
darwin.

I’ll study other arch/os on request.

François

[Jimmy Tang]

On Friday, 20 February 2015 20:20:05 UTC, François Bissey wrote:

Beware that the current stuff only applies to linux on intel/amd hardware.
I can add linux ppc easily and there is a patch to test things on recent
darwin.

I’ll study other arch/os on request.

François

Linux on arm v7 support for these patches would be nice ;)

Jimmy

 

[Francois Bissey]

So two parts:
--- aarch64.orig/aarch64-linux.h 2015-02-21 21:52:26.231219376 +1300
+++ aarch64/aarch64-linux.h 2015-02-21 21:55:26.121627263 +1300
@@ -32,6 +32,7 @@
%{rdynamic:-export-dynamic} \
-dynamic-linker " GNU_USER_DYNAMIC_LINKER " \
-X \
+ -rpath @@ARTIFACT@@/lib \
%{mbig-endian:-EB} %{mlittle-endian:-EL} \
-maarch64linux%{mbig-endian:b}”

And
--- arm.orig/linux-elf.h 2015-02-21 21:52:34.151237332 +1300
+++ arm/linux-elf.h 2015-02-21 21:57:50.681955230 +1300
@@ -72,6 +72,7 @@
%{rdynamic:-export-dynamic} \
-dynamic-linker " GNU_USER_DYNAMIC_LINKER "} \
-X \
+ -rpath @@ARTIFACT@@/lib \
%{mbig-endian:-EB} %{mlittle-endian:-EL}" \
SUBTARGET_EXTRA_LINK_SPEC

Unlike for x86 I don’t think it is necessary for it to come last.
All the other options after it are related to arch settings.
This will cover a lot if not all arm chips out there.

François

[Jimmy Tang]

oops, I hit reply instead of reply all


---------- Forwarded message ----------
From: Jimmy Tang <jcf...@gmail.com>
Date: Sat, Feb 21, 2015 at 9:22 AM
Subject: Re: [hashdist-group] Using gcc from Hashstack to compile
Hashstack (bootstrapping)
To: Francois Bissey <francoi...@canterbury.ac.nz>


I'll have to fire up my dev boards to try this next week. I'm
currently trying gcc on my mac right now, however there are some other
build issues on osx....

https://github.com/hashdist/hashstack/issues/661

On Sat, Feb 21, 2015 at 9:19 AM, Francois Bissey
<francoi...@canterbury.ac.nz> wrote:
> cortex-a9/armv7 I think. That’s the arm folder rather than aarch64 you are right.
> If the path for arm work the one for aarch64 should too.
>
> François
>
>> On 21/02/2015, at 22:04, Jimmy Tang <jcf...@gmail.com> wrote:
>>
>> Hi Francois,
>>
>>
>> On Sat, Feb 21, 2015 at 8:52 AM, Francois Bissey
>> <francoi...@canterbury.ac.nz> wrote:
>>> Hum… I will do both aarch64 and arm configuration as I am not certain
>>> which one it would be. From the look of it, it will cover a wide variety
>>> of arm chips.
>>> Do you have any idea what arch gcc you are internally?
>>>
>>> Francois
>>>
>>>
>>
>> The specific arm v7 chipset that I am interested in is this -
>> http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=i.MX6Q
>>
>> As far as I understand it, the v7 based chips are 32bit only, the
>> internal arch in gcc might be armv7 or armhf
>>
>> Jimmy
>



--
http://jcftang.github.io/


--
http://jcftang.github.io/

[Francois Bissey]

Just to be clear the OS X is without my experimental patch?

François

[Jimmy Tang]

It's with the patch, but the problem is a more generic issue with gcc 4.9.2 on OSX. I tried building gcc with out the rpath patches and it still fails.

[Ondřej Čertík]

Thanks Jimmy for trying it. I will leave it for you or anybody else to
first test the patch on OS X, before merging it, as I don't feel
comfortable to merge code that I cannot test (and that nobody else
tested).