MPI Error

[Barthle, Jonathan]

Good afternoon,

I am attempting to re-run a sensitivity analysis with RAVEN on Bitterroot. When I try to run it, I get the following error across my output files:

mpiexec: error while loading shared libraries: libhwloc.so.15: cannot open shared object file: No such file or directory

I have used the same RAVEN input in the past. I wanted to reach out and ask what might be causing this issue now. Attached is the xml file I am using. Let me know if there are any additional questions for me.

Thank you
Jonathan

[Joshua J. Cogliati]

Hm, my quick check on mpiexec on bitterroot:

salloc --wckey=nric --time=10:00:00 --cpus-per-task=30 --ntasks=1

srun --pty -I bash

module load openmpi

command -V mpiexec

ldd /apps/local/openmpi/5.0.5-gcc13.2.0-container/bin/mpiexec

      linux-vdso.so.1 (0x0000155555551000)

      libpthread.so.0 => /lib64/libpthread.so.0 (0x0000155555104000)

      librt.so.1 => /lib64/librt.so.1 (0x0000155554efc000)

      libpmix.so.2 => /apps/local/openmpi/5.0.5-gcc13.2.0-container/lib/libpmix.so.2 (0x0000155554acb000)

      libm.so.6 => /lib64/libm.so.6 (0x0000155554749000)

      libutil.so.1 => /lib64/libutil.so.1 (0x0000155554545000)

      libdl.so.2 => /lib64/libdl.so.2 (0x0000155554341000)

      libevent_core-2.1.so.7 => /apps/local/openmpi/5.0.5-gcc13.2.0-container/lib/libevent_core-2.1.so.7 (0x000015555410c000)

      libevent_pthreads-2.1.so.7 => /apps/local/openmpi/5.0.5-gcc13.2.0-container/lib/libevent_pthreads-2.1.so.7 (0x0000155553f09000)

      libhwloc.so.15 => /lib64/libhwloc.so.15 (0x0000155553cb9000)

      libc.so.6 => /lib64/libc.so.6 (0x00001555538e3000)

      /lib64/ld-linux-x86-64.so.2 (0x0000155555324000)

Shows libhwloc getting found in /lib64 so I am not sure which problem you are running into.

There is a "which mpiexec" in raven_ec_slurm_command.sh, so somewhere in the output should be the full path to mpiexec that is being run.

Sorry I don't have a better answer.

Joshua Cogliati

---

From: 'Barthle, Jonathan' via INL RAVEN Users Group <inl-rav...@googlegroups.com>
Sent: Wednesday, March 4, 2026 11:20 AM
To: inl-rav...@googlegroups.com <inl-rav...@googlegroups.com>
Subject: MPI Error

 

--
You received this message because you are subscribed to the Google Groups "INL RAVEN Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inl-raven-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/inl-raven-users/CAMsK15%2BByqpBMWEp%2BU%3DRcg0asD75BjG7OfDdM4dLwpJOXdZkkg%40mail.gmail.com.

[Barthle, Jonathan]

Hi Joshua,

Thank you for reaching out. I did some similar checks and also found libhwloc.so.15. I have attempted some troubleshooting on my side.  I think I may have found the issue. I am able to find /lib64/libhwloc.so.15 on the login and regular nodes on Bitterroot. 

I typically try to run these cases on the high bandwidth nodes. However, when I try to find it on the high bandwidth node, I get: ls: cannot access '/lib64/libhwloc.so.15': No such file or directory. When I switch back over to normal nodes, it works just fine. Therefore, this seems like an issue only with the high bandwidth nodes.

Thank you

Jonathan