SIGSEGV: Segmentation fault using plumed and pmemd.cuda.MPI in AMBER22/AMBER20
378 views
Skip to first unread message
Ciara Wallis
Aug 10, 2022, 3:39:34 AM8/10/22
to PLUMED users
Hi everyone,
I am currently trying to run some replica exchange umbrella sampling simulations with AMBER22 using pmemd.cuda.MPI and plumed. So far, I have been able to get this to work on pmemd.MPI with plumed, and on pmemd.cuda.MPI with plumed turned off, but when I try and run pmemd.cuda.MPI with plumed I get the error message below:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
I've also tried using AMBER20 with plumed to do REUS, and get the same seg fault when using pmemd.cuda.MPI.
I was wondering if anyone else had experienced a similar issue trying to run pmemd.cuda.MPI with plumed, and if anyone could please give any advice on how to not get this seg fault error.
Thank you very much in advance.
Kindest regards,
Ciara
I am currently trying to run some replica exchange umbrella sampling simulations with AMBER22 using pmemd.cuda.MPI and plumed. So far, I have been able to get this to work on pmemd.MPI with plumed, and on pmemd.cuda.MPI with plumed turned off, but when I try and run pmemd.cuda.MPI with plumed I get the error message below:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
I've also tried using AMBER20 with plumed to do REUS, and get the same seg fault when using pmemd.cuda.MPI.
I was wondering if anyone else had experienced a similar issue trying to run pmemd.cuda.MPI with plumed, and if anyone could please give any advice on how to not get this seg fault error.
Thank you very much in advance.
Kindest regards,
Ciara
Here's a copy of the whole error message:
Loading amber/22
Loading requirement: openmpi/4.1.3 intel-mkl/2021.4.0 python3/3.10.4
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Backtrace for this error:
#0 0x1546f604f171 in ???
#1 0x1546f604e313 in ???
#2 0x1546f5860b1f in ???
#0 0x1497e681c171 in ???
#0 0x14daa80cc171 in ???
#1 0x14daa80cb313 in ???
#2 0x14daa78ddb1f in ???
#0 0x14c80a8bf171 in ???
#1 0x14c80a8be313 in ???
#2 0x14c80a0d0b1f in ???
#0 0x15412c61d171 in ???
#1 0x15412c61c313 in ???
#2 0x15412be2eb1f in ???
#0 0x15509efd2171 in ???
#1 0x15509efd1313 in ???
#0 0x14eade98b171 in ???
#1 0x14eade98a313 in ???
#0 0x153b33f5d171 in ???
#1 0x153b33f5c313 in ???
#2 0x153b3376eb1f in ???
#2 0x15509e7e3b1f in ???
#0 0x14def3582171 in ???
#1 0x14def3581313 in ???
#2 0x14eade19cb1f in ???
#2 0x14def2d93b1f in ???
#1 0x1497e681b313 in ???
#0 0x1470a8dd7171 in ???
#2 0x1497e602db1f in ???
#0 0x148817e8a171 in ???
#1 0x1470a8dd6313 in ???
#2 0x1470a85e8b1f in ???
#1 0x148817e89313 in ???
#2 0x14881769bb1f in ???
#3 0x154134b86be1 in ???
#3 0x153b3c4c6be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14dab0635be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14c812e28be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14eae6ef4be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1546fe5b8be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1550a753bbe1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14defbaebbe1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1497eed85be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1470b1340be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1488203f3be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#0 0x14f4f7cdf171 in ???
#1 0x14f4f7cde313 in ???
#2 0x14f4f74f0b1f in ???
#3 0x14f500248be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#4 0x14de85408944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14de85408944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14ea2f6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14ea2f6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x15465b6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x15465b6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x155030cb6944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x155030cb6944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x153a896f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x153a896f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14d9fc99b944 in _ZN4PLMD12Communicator3SumIiEEvRT_
#4 0x14c75b6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14c75b6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x146ffd6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x146ffd6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
at tools/../../tools/Communicator.h:179
#5 0x14d9fc99b944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14877d6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
#4 0x15408099b944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x15408099b944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14974b6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14974b6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
at tools/../../tools/Communicator.h:179
#5 0x14877d6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14f489126944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14f489126944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#6 0x14d9fc9b28e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14877d70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14c75b70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x1540809b28e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
#6 0x14ea2f70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14974b70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x15465b70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x155030ccd8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x153a8970e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x146ffd70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14f48913d8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14de8541f8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#7 0x146ffd71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x146ffd71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14d9fc9bf6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14d9fc9bf6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14ea2f71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14ea2f71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14974b71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
#7 0x15465b71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x15465b71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#7 0x153a8971b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x153a8971b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#7 0x1540809bf6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x1540809bf6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#7 0x14877d71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14877d71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#7 0x14c75b71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14c75b71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14f48914a6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14f48914a6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14de8542c6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14de8542c6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#7 0x155030cda6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x155030cda6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x153b3375acf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x15412be1acf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x1470a85d4cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x148817687cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14c80a0bccf2 in ???
#17 0x4eb0ed in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14def2d7fcf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14eade188cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14daa78c9cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x15509e7cfcf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x1546f584ccf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#18 0xffffffffffffffff in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14f4f74dccf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14974b71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x1497e6019cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 0 on node gadi-gpu-v100-0083 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
Loading requirement: openmpi/4.1.3 intel-mkl/2021.4.0 python3/3.10.4
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
+++ Loading the PLUMED kernel runtime +++
+++ PLUMED_KERNEL="/apps/plumed/2.7.3/lib/libplumedKernel.so" +++
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Backtrace for this error:
#0 0x1546f604f171 in ???
#1 0x1546f604e313 in ???
#2 0x1546f5860b1f in ???
#0 0x1497e681c171 in ???
#0 0x14daa80cc171 in ???
#1 0x14daa80cb313 in ???
#2 0x14daa78ddb1f in ???
#0 0x14c80a8bf171 in ???
#1 0x14c80a8be313 in ???
#2 0x14c80a0d0b1f in ???
#0 0x15412c61d171 in ???
#1 0x15412c61c313 in ???
#2 0x15412be2eb1f in ???
#0 0x15509efd2171 in ???
#1 0x15509efd1313 in ???
#0 0x14eade98b171 in ???
#1 0x14eade98a313 in ???
#0 0x153b33f5d171 in ???
#1 0x153b33f5c313 in ???
#2 0x153b3376eb1f in ???
#2 0x15509e7e3b1f in ???
#0 0x14def3582171 in ???
#1 0x14def3581313 in ???
#2 0x14eade19cb1f in ???
#2 0x14def2d93b1f in ???
#1 0x1497e681b313 in ???
#0 0x1470a8dd7171 in ???
#2 0x1497e602db1f in ???
#0 0x148817e8a171 in ???
#1 0x1470a8dd6313 in ???
#2 0x1470a85e8b1f in ???
#1 0x148817e89313 in ???
#2 0x14881769bb1f in ???
#3 0x154134b86be1 in ???
#3 0x153b3c4c6be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14dab0635be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14c812e28be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14eae6ef4be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1546fe5b8be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1550a753bbe1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x14defbaebbe1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1497eed85be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1470b1340be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#3 0x1488203f3be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#0 0x14f4f7cdf171 in ???
#1 0x14f4f7cde313 in ???
#2 0x14f4f74f0b1f in ???
#3 0x14f500248be1 in ???
at /jobfs/37634155.gadi-pbs/0/openmpi/4.1.3/source/openmpi-4.1.3/opal/class/opal_object.h:481
#4 0x14de85408944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14de85408944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14ea2f6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14ea2f6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x15465b6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x15465b6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x155030cb6944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x155030cb6944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x153a896f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x153a896f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14d9fc99b944 in _ZN4PLMD12Communicator3SumIiEEvRT_
#4 0x14c75b6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14c75b6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x146ffd6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x146ffd6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
at tools/../../tools/Communicator.h:179
#5 0x14d9fc99b944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14877d6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
#4 0x15408099b944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x15408099b944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14974b6f7944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14974b6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
at tools/../../tools/Communicator.h:179
#5 0x14877d6f7944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#4 0x14f489126944 in _ZN4PLMD12Communicator3SumIiEEvRT_
at tools/../../tools/Communicator.h:179
#5 0x14f489126944 in _ZN4PLMD4GREX3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/GREX.cpp:103
#6 0x14d9fc9b28e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14877d70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14c75b70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x1540809b28e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
#6 0x14ea2f70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14974b70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x15465b70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x155030ccd8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x153a8970e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x146ffd70e8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14f48913d8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#6 0x14de8541f8e3 in _ZN4PLMD10PlumedMain3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMain.cpp:550
#7 0x146ffd71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x146ffd71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14d9fc9bf6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14d9fc9bf6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14ea2f71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14ea2f71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14974b71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
#7 0x15465b71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x15465b71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#7 0x153a8971b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x153a8971b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#7 0x1540809bf6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x1540809bf6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#7 0x14877d71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14877d71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#7 0x14c75b71b6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14c75b71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14f48914a6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14f48914a6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#7 0x14de8542c6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14de8542c6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#7 0x155030cda6b7 in _ZN4PLMD7WithCmd3cmdERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKv
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x155030cda6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x153b3375acf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x15412be1acf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x1470a85d4cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x148817687cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14c80a0bccf2 in ???
#17 0x4eb0ed in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14def2d7fcf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14eade188cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14daa78c9cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x15509e7cfcf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x1546f584ccf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
#18 0xffffffffffffffff in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x14f4f74dccf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/WithCmd.h:46
#8 0x14974b71b6b7 in plumed_plumedmain_cmd
at /jobfs/35162560.gadi-pbs/0/plumed/2.7.3/source/plumed-2.7.3/src/core/PlumedMainInitializer.cpp:67
#9 0x70c1b9 in ???
#10 0x70c5d1 in ???
#11 0x70d52c in ???
#12 0x70d46e in ???
#13 0x591852 in ???
#14 0x5d9ff1 in ???
#15 0x4d182c in ???
#16 0x1497e6019cf2 in ???
#17 0x4eb0ed in ???
#18 0xffffffffffffffff in ???
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 0 on node gadi-gpu-v100-0083 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
Tien Phan
Mar 28, 2023, 3:19:03 PM3/28/23
to PLUMED users
Hi Ciara,
Have you figured out how to get PLUMED to work with pmemd.cuda.MPI? I also got the same error when attempting to use pmemd.cuda.MPI in Amber22 to run Parallel Tempering - Well Tempered Ensemble (PT-WTE) with plumed 2.8.2. In my case, it also works just fine with pmemd.MPI but plumed throws the Segmentation fault error when running with pmemd.cuda.MPI.
I appreciate it if anyone knows how to fix this error.
Thanks,
Tien
Reply all
Reply to author
Forward
0 new messages