[slurm-users] FreeBSD/aarch64: ld: error: unknown emulation: elf_aarch64

[Nuno Teixeira via slurm-users]

Hello,

I'm new maintainer of slurm FreeBSD port and for a long time it failing on aarch64 platform.

Any clues?

> ld: error: unknown emulation: elf_aarch64

`clang -print-targets`:

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_32 - AArch64 (little endian ILP32)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    arm64_32   - ARM64 (little endian ILP32)
    armeb      - ARM (big endian)
    ppc32      - PowerPC 32
    ppc32le    - PowerPC 32 LE
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    riscv32    - 32-bit RISC-V
    riscv64    - 64-bit RISC-V
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64

Full log:

https://pkg-status.freebsd.org/ampere2/data/main-arm64-default/pb78025a96ed9_s3c1be0b2615/logs/slurm-wlm-23.11.6.log

Thanks,

--

Nuno Teixeira

FreeBSD UNIX:  <edu...@FreeBSD.org>   Web:  https://FreeBSD.org

[Christopher Samuel via slurm-users]

On 5/4/24 4:24 am, Nuno Teixeira via slurm-users wrote:

> Any clues?
>
> > ld: error: unknown emulation: elf_aarch64

All I can think is that your ld doesn't like elf_aarch64, from the log
your posting it looks that's being injected from the FreeBSD ports
system. Looking at the man page for ld on Linux it says:

-m emulation
Emulate the emulation linker. You can list the available
emulations with the --verbose or -V options.

So I'd guess you'd need to look at what that version of ld supports and
then update the ports system to match.

Good luck!

All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA


--
slurm-users mailing list -- slurm...@lists.schedmd.com
To unsubscribe send an email to slurm-us...@lists.schedmd.com

[Nuno Teixeira via slurm-users]

Hello,

I instructed port to use binutils from ports (version 2.40 native) instead of base:

`/usr/local/bin/ld: unrecognised emulation mode: elf_aarch64`

```

/usr/local/bin/ld -V |grep aarch64
   aarch64cloudabi
   aarch64cloudabib
   aarch64elf
   aarch64elf32
   aarch64elf32b
   aarch64elfb
   aarch64fbsd
   aarch64fbsdb
   aarch64haiku
   aarch64linux
   aarch64linux32
   aarch64linux32b
   aarch64linuxb
   aarch64pe

```

Any clues about "elf_aarch64" and "aarch64elf" mismatch?

Thanks,

[Steffen Grunewald via slurm-users]

On Mon, 2024-05-06 at 11:38:30 +0100, Slurm users wrote:
> Hello,
>
> I instructed port to use binutils from ports (version 2.40 native) instead
> of base:
>
> `/usr/local/bin/ld: unrecognised emulation mode: elf_aarch64`
>
> ```
> /usr/local/bin/ld -V |grep aarch64
> aarch64cloudabi
> aarch64cloudabib
> aarch64elf
> aarch64elf32
> aarch64elf32b
> aarch64elfb
> aarch64fbsd
> aarch64fbsdb
> aarch64haiku
> aarch64linux
> aarch64linux32
> aarch64linux32b
> aarch64linuxb
> aarch64pe
> ```
>
> Any clues about "elf_aarch64" and "aarch64elf" mismatch?

This looks (I admit, I haven't UTSL) like the emulation mode is constructed
from an "elf_" prefix and the architecture nickname - this works for "x86_64"
and "i386" since the "ld" for the Intel/AMD architectures indeed provides the
emulations "elf_x86_64" and "elf_i386" while for 64-bit ARM "elf" is used as
a suffix. So this is mainly an ld inconsistency, I'm afraid (which might be
fixed by adding alias names - but the hopes are pretty low).

Non-emulated builds shouldn't be affected by the issue you found, right?
(There is Slurm built for ARM64 Debian. Maybe they have patched the source?)


Two ways to get this fixed I can imagine:
(a) find the place where the emulation mode name is combined, and teach that
of possible exceptions to the implemented rule (there may be more than just
ARM - what about RISC-V, PPC64*, ...?)
(b) interrupt the build in a reasonable place, find all occurreences of the
wrong emulation string, and replace it with its existing counterpart

There should be no doubt which one I'd prefer - I'll go and read TS ;)

Cheers,
Steffen


--
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~

[Christopher Samuel via slurm-users]

On 5/6/24 6:38 am, Nuno Teixeira via slurm-users wrote:

> Any clues about "elf_aarch64" and "aarch64elf" mismatch?

As I mentioned I think this is coming from the FreeBSD patching that's
being done to the upstream Slurm sources, specifically it looks like
elf_aarch64 is being injected here:

/usr/bin/sed -i.bak -e 's|"/proc|"/compat/linux/proc|g' -e
's|(/proc)|(/compat/linux/proc)|g'
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/slurmd/slurmstepd/req.c
/usr/bin/find
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/api
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/plugins/openapi
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/sacctmgr
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/sackd
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/scontrol
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/scrontab
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/scrun
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/slurmctld
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/slurmd/slurmd
/wrkdirs/usr/ports/sysutils/slurm-wlm/work/slurm-23.11.6/src/squeue
-name Makefile.in | /usr/bin/xargs /usr/bin/sed -i.bak -e 's|-r -o|-r
-m elf_aarch64 -o|'

So I guess that will need to be fixed to match what FreeBSD supports.

I don't think this is a Slurm issue from what I see there.

[Nuno Teixeira via slurm-users]

Hello,

I too think this the cause and I really missed it:

.if ${ARCH} == powerpc64le
        ${FIND} ${LLD2FIX:C|^|${WRKSRC}/src/|} -name Makefile.in | ${XARGS}     \
                ${REINPLACE_CMD} -e 's|-r -o|-r -m elf64lppc -o|'
.elif ${ARCH} == powerpc64
        ${FIND} ${LLD2FIX:C|^|${WRKSRC}/src/|} -name Makefile.in | ${XARGS}     \
                ${REINPLACE_CMD} -e 's|-r -o|-r -m elf64ppc -o|'
.else
        ${FIND} ${LLD2FIX:C|^|${WRKSRC}/src/|} -name Makefile.in | ${XARGS}     \
                ${REINPLACE_CMD} -e 's|-r -o|-r -m elf_${ARCH} -o|'

I will adjust it and see build result.

Thanks,

[Nuno Teixeira via slurm-users]

(...)

Fixed with:

+.elif ${ARCH} == aarch64
+       ${FIND} ${LLD2FIX:C|^|${WRKSRC}/src/|} -name Makefile.in | ${XARGS}     \
+               ${REINPLACE_CMD} -e 's|-r -o|-r -m aarch64elf -o|'

Thanks and sorry for the noise as I really missed this detail :)

Cheers,

[Christopher Samuel via slurm-users]

On 5/6/24 3:19 pm, Nuno Teixeira via slurm-users wrote:

> Fixed with:
>
[...]

>
> Thanks and sorry for the noise as I really missed this detail :)

So glad it helped! Best of luck with this work.