GRAPHLAB_SUBNET_ID/GRAPHLAB_SUBNET_MASK environment variables not defined. Using default values Subnet ID: 0.0.0.0 Subnet Mask: 0.0.0.0 Will find first IPv4 non-loopback address matching the subnet tcmalloc: large alloc 5066549580800000 bytes == (nil) @ 0x65b851 0x7f9f1d417a89 tcmalloc: large alloc 5348024557510656 bytes == (nil) @ 0x65b851 0x7f9d8bf47a89 tcmalloc: large alloc 3759995725912481792 bytes == (nil) @ 0x65b851 0x7fa9b084ca89 terminate called after throwing an instance of 'std::bad_alloc' tcmalloc: large alloc 5348024557510656 bytes == (nil) @ 0x65b851 0x7f0dbf228a89 ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) tcmalloc: large alloc 3472556787679371264 bytes == (nil) @ 0x65b851 0x7f91e13dfa89 ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) terminate called after throwing an instance of 'std::bad_alloc' tcmalloc: large alloc 3530822107858477056 bytes == (nil) @ 0x65b851 0x7fdffe09fa89 tcmalloc: large alloc 3530822107858477056 bytes == (nil) @ 0x65b851 0x7f90fcd2aa89 ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) tcmalloc: large alloc 5348024557510656 bytes == (nil) @ 0x65b851 0x7fe0dde7fa89 ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) terminate called after throwing an instance of ' what(): std::bad_alloc tcmalloc: large alloc 3472556787679371264 bytes == (nil) @ 0x65b851 0x7f75d7ccea89 ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) tcmalloc: large alloc 3530822107858477056 bytes == (nil) @ 0x65b851 0x7f24d966ea89 tcmalloc: large alloc 3328214000696565760 bytes == (nil) @ 0x65b851 0x7fe5b45a9a89 tcmalloc: large alloc 3543822943798697984 bytes == (nil) @ 0x65b851 0x7f4210902a89 ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) tcmalloc: large alloc 3759995725912481792 bytes == (nil) @ 0x65b851 0x7f8183724a89 tcmalloc: large alloc 3472556787679371264 bytes == (nil) @ 0x65b851 0x7f16a1a76a89 tcmalloc: large alloc 3905239000875810816 bytes == (nil) @ 0x65b851 0x7f7b00cf7a89 tcmalloc: large alloc 3530822107858477056 bytes == (nil) @ 0x65b851 0x7fc71b620a89 terminate called after throwing an instance of 'std::bad_alloc' tcmalloc: large alloc 3530822107858477056 bytes == (nil) @ 0x65b851 0x7fa8888f0a89 tcmalloc: large alloc 3530822107858477056 bytes == (nil) @ 0x65b851 0x7f77f656fa89 [node006:02419] *** Process received signal *** terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc [node019:02426] *** Process received signal *** what(): std::bad_alloc terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' [node053:02219] *** Process received signal *** [node053:02219] Signal: Aborted (6) [node053:02219] Signal code: (-6) std::bad_alloc' [node059:02144] *** Process received signal *** terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' [node081:02117] *** Process received signal *** terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc [node082:02129] *** Process received signal *** [node082:02129] Signal: Aborted (6) [node082:02129] Signal code: (-6) [node083:02145] *** Process received signal *** [node083:02145] Signal: Aborted (6) [node083:02145] Signal code: (-6) terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' [node006:02419] Signal: Aborted (6) [node006:02419] Signal code: (-6) ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) [node016:02354] *** Process received signal *** [node016:02354] Signal: Aborted (6) [node016:02354] Signal code: (-6) [node019:02426] Signal: Aborted (6) [node019:02426] Signal code: (-6) [node015:02378] *** Process received signal *** [node020:02433] *** Process received signal *** [node020:02433] Signal: Aborted (6) [node020:02433] Signal code: (-6) what(): std::bad_alloc [node021:02311] *** Process received signal *** [node017:02375] *** Process received signal *** [node017:02375] Signal: Aborted (6) [node017:02375] Signal code: (-6) what(): std::bad_alloc [node060:02148] *** Process received signal *** [node053:02219] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f8046018cb0] [node053:02219] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f8043528425] [node053:02219] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f804352bb8b] [node053:02219] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node053:02219] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node053:02219] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] what(): std::bad_alloc [node059:02144] Signal: Aborted (6) [node059:02144] Signal code: (-6) what(): std::bad_alloc [node064:02188] *** Process received signal *** what(): std::bad_alloc what(): std::bad_alloc what(): std::bad_alloc [node081:02117] Signal: Aborted (6) [node081:02117] Signal code: (-6) what(): std::bad_alloc ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) what(): std::bad_alloc what(): std::bad_alloc ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) what(): std::bad_alloc [node082:02129] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f0dc16dbcb0] [node082:02129] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f0dbebeb425] [node082:02129] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f0dbebeeb8b] [node082:02129] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f0dbf24169d] [node082:02129] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f0dbf23f846] [node082:02129] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f0dbf23f873] [node082:02129] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f0dbf23f96e] [node082:02129] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node082:02129] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f0dbf228a89] [node082:02129] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f0dbf2297fb] [node082:02129] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f0dbf2298e0] [node083:02145] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fdcf6604cb0] [node083:02145] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fdcf3b14425] [node083:02145] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fdcf3b17b8b] [node083:02145] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node083:02145] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node083:02145] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node083:02145] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fdcf3aff76d] what(): std::bad_alloc what(): std::bad_alloc [node006:02419] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7febda358cb0] [node006:02419] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7febd7868425] [node006:02419] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7febd786bb8b] [node006:02419] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node006:02419] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node006:02419] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node006:02419] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7febd785376d] [node006:02419] [ 7] ./gibbs-gmm() [0x4aedc9] [node007:02341] *** Process received signal *** ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) [node016:02354] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f91e3892cb0] [node016:02354] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f91e0da2425] [node016:02354] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f91e0da5b8b] [node016:02354] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f91e13f869d] [node016:02354] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f91e13f6846] [node016:02354] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f91e13f6873] [node016:02354] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f91e13f696e] [node016:02354] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node016:02354] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f91e13dfa89] [node016:02354] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f91e13e07fb] [node016:02354] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f91e13e08e0] [node016:02354] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f91e13e0a3f] [node019:02426] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f38d091acb0] [node019:02426] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f38cde2a425] [node019:02426] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f38cde2db8b] [node019:02426] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node019:02426] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node019:02426] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node019:02426] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f38cde1576d] [node019:02426] [ 7] ./gibbs-gmm() [0x4aedc9] [node015:02378] Signal: Aborted (6) [node015:02378] Signal code: (-6) [node020:02433] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fe000552cb0] [node020:02433] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fdffda62425] [node020:02433] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fdffda65b8b] [node020:02433] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fdffe0b869d] [node020:02433] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fdffe0b6846] [node020:02433] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fdffe0b6873] [node020:02433] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fdffe0b696e] [node020:02433] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node021:02311] Signal: Aborted (6) [node021:02311] Signal code: (-6) [node017:02375] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f8b09a00cb0] [node017:02375] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f8b06f10425] [node017:02375] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f8b06f13b8b] [node017:02375] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node017:02375] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node017:02375] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node017:02375] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f8b06efb76d] [node017:02375] [ 7] ./gibbs-gmm() [0x4aedc9] [node017:02375] *** End of error message *** [node060:02148] Signal: Aborted (6) [node060:02148] Signal code: (-6) [node053:02219] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f804351376d] [node053:02219] [ 7] ./gibbs-gmm() [0x4aedc9] [node053:02219] *** End of error message *** [node061:02230] *** Process received signal *** [node059:02144] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f9f1f8cacb0] [node059:02144] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f9f1cdda425] [node059:02144] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f9f1cdddb8b] [node059:02144] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f9f1d43069d] [node059:02144] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f9f1d42e846] [node059:02144] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f9f1d42e873] [node059:02144] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f9f1d42e96e] [node059:02144] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node068:02208] *** Process received signal *** [node064:02188] Signal: Aborted (6) [node064:02188] Signal code: (-6) [node066:02221] *** Process received signal *** [node066:02221] Signal: Aborted (6) [node066:02221] Signal code: (-6) [node067:02220] *** Process received signal *** [node069:02130] *** Process received signal *** [node081:02117] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f813c30acb0] [node081:02117] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f813981a425] [node081:02117] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f813981db8b] [node081:02117] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node081:02117] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node081:02117] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node081:02117] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f813980576d] [node081:02117] [ 7] ./gibbs-gmm() [0x4aedc9] [node079:02115] *** Process received signal *** [node079:02115] Signal: Aborted (6) [node079:02115] Signal code: (-6) [node080:02351] *** Process received signal *** [node077:02219] *** Process received signal *** [node072:02150] *** Process received signal *** [node072:02150] Signal: Aborted (6) [node072:02150] Signal code: (-6) [node071:02193] *** Process received signal *** [node052:02155] *** Process received signal *** [node052:02155] Signal: Aborted (6) [node052:02155] Signal code: (-6) [node082:02129] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f0dbf229a3f] [node082:02129] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node082:02129] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node082:02129] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node082:02129] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f0dbebd676d] [node082:02129] [16] ./gibbs-gmm() [0x4aedc9] [node082:02129] *** End of error message *** [node083:02145] [ 7] ./gibbs-gmm() [0x4aedc9] [node083:02145] *** End of error message *** [node084:02188] *** Process received signal *** [node085:02135] *** Process received signal *** [node006:02419] *** End of error message *** [node007:02341] Signal: Aborted (6) [node007:02341] Signal code: (-6) [node009:02369] *** Process received signal *** [node008:02429] *** Process received signal *** [node008:02429] Signal: Aborted (6) [node008:02429] Signal code: (-6) [node016:02354] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node016:02354] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node016:02354] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node016:02354] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f91e0d8d76d] [node016:02354] [16] ./gibbs-gmm() [0x4aedc9] [node016:02354] *** End of error message *** [node019:02426] *** End of error message *** [node015:02378] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fa9b2cffcb0] [node015:02378] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fa9b020f425] [node015:02378] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fa9b0212b8b] [node015:02378] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fa9b086569d] [node015:02378] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fa9b0863846] [node015:02378] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fa9b0863873] [node020:02433] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fdffe09fa89] [node020:02433] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fdffe0a07fb] [node020:02433] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fdffe0a08e0] [node020:02433] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fdffe0a0a3f] [node020:02433] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node020:02433] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node020:02433] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node020:02433] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fdffda4d76d] [node020:02433] [16] ./gibbs-gmm() [0x4aedc9] [node020:02433] *** End of error message *** [node021:02311] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f90ff1ddcb0] [node021:02311] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f90fc6ed425] [node021:02311] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f90fc6f0b8b] [node021:02311] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f90fcd4369d] [node021:02311] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f90fcd41846] [node021:02311] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f90fcd41873] [node021:02311] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f90fcd4196e] [node021:02311] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node021:02311] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f90fcd2aa89] [node021:02311] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f90fcd2b7fb] [node021:02311] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f90fcd2b8e0] [node021:02311] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f90fcd2ba3f] [node021:02311] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node060:02148] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fe0e0332cb0] [node060:02148] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fe0dd842425] [node060:02148] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fe0dd845b8b] [node060:02148] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fe0dde9869d] [node060:02148] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fe0dde96846] [node060:02148] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fe0dde96873] [node060:02148] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fe0dde9696e] [node060:02148] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node060:02148] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fe0dde7fa89] [node061:02230] Signal: Aborted (6) [node061:02230] Signal code: (-6) [node059:02144] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f9f1d417a89] [node059:02144] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f9f1d4187fb] [node059:02144] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f9f1d4188e0] [node059:02144] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f9f1d418a3f] [node059:02144] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node059:02144] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node059:02144] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node059:02144] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f9f1cdc576d] [node059:02144] [16] ./gibbs-gmm() [0x4aedc9] [node059:02144] *** End of error message *** [node068:02208] Signal: Aborted (6) [node068:02208] Signal code: (-6) [node064:02188] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f02dce31cb0] [node064:02188] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f02da341425] [node064:02188] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f02da344b8b] [node064:02188] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node064:02188] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node064:02188] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node066:02221] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f24dbb21cb0] [node066:02221] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f24d9031425] [node066:02221] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f24d9034b8b] [node066:02221] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f24d968769d] [node066:02221] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f24d9685846] [node066:02221] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f24d9685873] [node066:02221] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f24d968596e] [node066:02221] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node066:02221] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f24d966ea89] [node067:02220] Signal: Aborted (6) [node067:02220] Signal code: (-6) [node069:02130] Signal: Aborted (6) [node069:02130] Signal code: (-6) [node081:02117] *** End of error message *** [node079:02115] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f8185bd7cb0] [node079:02115] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f81830e7425] [node079:02115] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f81830eab8b] [node079:02115] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f818373d69d] [node079:02115] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f818373b846] [node079:02115] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f818373b873] [node079:02115] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f818373b96e] [node079:02115] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node073:02213] *** Process received signal *** [node080:02351] Signal: Aborted (6) [node080:02351] Signal code: (-6) [node077:02219] Signal: Aborted (6) [node077:02219] Signal code: (-6) [node072:02150] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f76c6786cb0] [node072:02150] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f76c3c96425] [node072:02150] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f76c3c99b8b] [node072:02150] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node072:02150] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node072:02150] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node072:02150] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f76c3c8176d] [node071:02193] Signal: Aborted (6) [node071:02193] Signal code: (-6) [node052:02155] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fc71dad3cb0] [node052:02155] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fc71afe3425] [node052:02155] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fc71afe6b8b] [node052:02155] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fc71b63969d] [node052:02155] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fc71b637846] [node052:02155] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fc71b637873] [node052:02155] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fc71b63796e] [node052:02155] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node052:02155] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fc71b620a89] [node084:02188] Signal: Aborted (6) [node084:02188] Signal code: (-6) [node085:02135] Signal: Aborted (6) [node085:02135] Signal code: (-6) [node007:02341] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f223b111cb0] [node007:02341] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f2238621425] [node007:02341] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f2238624b8b] [node007:02341] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node007:02341] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node007:02341] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node007:02341] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f223860c76d] [node009:02369] Signal: Aborted (6) [node009:02369] Signal code: (-6) [node008:02429] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f106bdb7cb0] [node008:02429] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f10692c7425] [node008:02429] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f10692cab8b] [node008:02429] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node008:02429] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node008:02429] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node008:02429] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f10692b276d] [node008:02429] [ 7] ./gibbs-gmm() [0x4aedc9] [node008:02429] *** End of error message *** [node015:02378] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fa9b086396e] [node015:02378] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node015:02378] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fa9b084ca89] [node015:02378] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fa9b084d7fb] [node015:02378] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fa9b084d8e0] [node015:02378] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fa9b084da3f] [node015:02378] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node015:02378] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node015:02378] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node015:02378] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa9b01fa76d] [node015:02378] [16] ./gibbs-gmm() [0x4aedc9] tcmalloc: large alloc 5348024557510656 bytes == (nil) @ 0x65b851 0x7fd1788e3a89 [node021:02311] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node021:02311] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node021:02311] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f90fc6d876d] [node021:02311] [16] ./gibbs-gmm() [0x4aedc9] [node021:02311] *** End of error message *** tcmalloc: large alloc 3328214000696565760 bytes == (nil) @ 0x65b851 0x7f03ee432a89 [node060:02148] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fe0dde807fb] [node060:02148] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fe0dde808e0] [node060:02148] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fe0dde80a3f] [node060:02148] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node060:02148] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node060:02148] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node060:02148] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fe0dd82d76d] [node060:02148] [16] ./gibbs-gmm() [0x4aedc9] [node060:02148] *** End of error message *** [node061:02230] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f9d8e3facb0] [node061:02230] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f9d8b90a425] [node061:02230] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f9d8b90db8b] [node061:02230] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f9d8bf6069d] [node061:02230] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f9d8bf5e846] [node061:02230] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f9d8bf5e873] [node061:02230] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f9d8bf5e96e] [node061:02230] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node068:02208] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f75da181cb0] [node068:02208] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f75d7691425] [node068:02208] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f75d7694b8b] [node068:02208] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f75d7ce769d] [node068:02208] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f75d7ce5846] [node068:02208] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f75d7ce5873] [node068:02208] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f75d7ce596e] [node068:02208] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node064:02188] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f02da32c76d] [node064:02188] [ 7] ./gibbs-gmm() [0x4aedc9] [node064:02188] *** End of error message *** [node066:02221] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f24d966f7fb] [node066:02221] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f24d966f8e0] [node066:02221] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f24d966fa3f] [node066:02221] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node066:02221] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node066:02221] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node066:02221] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f24d901c76d] [node066:02221] [16] ./gibbs-gmm() [0x4aedc9] [node066:02221] *** End of error message *** [node067:02220] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fe5b6a5ccb0] [node067:02220] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fe5b3f6c425] [node067:02220] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fe5b3f6fb8b] [node067:02220] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fe5b45c269d] [node067:02220] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fe5b45c0846] [node067:02220] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fe5b45c0873] [node067:02220] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fe5b45c096e] [node069:02130] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f4212db5cb0] [node069:02130] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f42102c5425] [node069:02130] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f42102c8b8b] [node069:02130] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f421091b69d] [node069:02130] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f4210919846] [node069:02130] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f4210919873] [node079:02115] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f8183724a89] [node079:02115] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f81837257fb] [node079:02115] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f81837258e0] [node079:02115] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f8183725a3f] [node079:02115] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node079:02115] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node079:02115] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node079:02115] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f81830d276d] [node079:02115] [16] ./gibbs-gmm() [0x4aedc9] [node079:02115] *** End of error message *** [node073:02213] Signal: Aborted (6) [node073:02213] Signal code: (-6) [node080:02351] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f16a3f29cb0] [node080:02351] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f16a1439425] [node080:02351] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f16a143cb8b] [node080:02351] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f16a1a8f69d] [node080:02351] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f16a1a8d846] ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) tcmalloc: large alloc 3328214000696565760 bytes == (nil) @ 0x65b851 0x7f2d6c3a7a89 [node077:02219] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f7b031aacb0] [node077:02219] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f7b006ba425] [node077:02219] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f7b006bdb8b] [node077:02219] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f7b00d1069d] [node077:02219] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f7b00d0e846] [node077:02219] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f7b00d0e873] [node077:02219] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f7b00d0e96e] [node077:02219] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node072:02150] [ 7] ./gibbs-gmm() [0x4aedc9] [node072:02150] *** End of error message *** [node071:02193] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fdaf3e38cb0] [node071:02193] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fdaf1348425] [node071:02193] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fdaf134bb8b] [node071:02193] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node071:02193] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node071:02193] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node071:02193] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fdaf133376d] [node071:02193] [ 7] ./gibbs-gmm() [0x4aedc9] [node052:02155] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fc71b6217fb] [node052:02155] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fc71b6218e0] [node052:02155] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fc71b621a3f] [node052:02155] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node052:02155] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node052:02155] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node052:02155] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fc71afce76d] [node052:02155] [16] ./gibbs-gmm() [0x4aedc9] [node052:02155] *** End of error message *** [node084:02188] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fa88ada3cb0] [node084:02188] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fa8882b3425] [node084:02188] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fa8882b6b8b] [node084:02188] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fa88890969d] [node084:02188] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fa888907846] [node084:02188] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fa888907873] [node085:02135] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f77f8a22cb0] [node085:02135] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f77f5f32425] [node085:02135] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f77f5f35b8b] [node085:02135] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f77f658869d] [node085:02135] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f77f6586846] [node085:02135] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f77f6586873] [node007:02341] [ 7] ./gibbs-gmm() [0x4aedc9] [node007:02341] *** End of error message *** [node009:02369] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f9295e06cb0] [node009:02369] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f9293316425] [node009:02369] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f9293319b8b] [node009:02369] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node009:02369] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node009:02369] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node015:02378] *** End of error message *** terminate called after throwing an instance of 'std::bad_alloc' terminate called after throwing an instance of 'std::bad_alloc' [node061:02230] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f9d8bf47a89] [node061:02230] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f9d8bf487fb] [node061:02230] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f9d8bf488e0] [node061:02230] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f9d8bf48a3f] [node061:02230] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node061:02230] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node061:02230] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node061:02230] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f9d8b8f576d] [node061:02230] [16] ./gibbs-gmm() [0x4aedc9] [node061:02230] *** End of error message *** [node068:02208] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f75d7ccea89] [node068:02208] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f75d7ccf7fb] [node068:02208] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f75d7ccf8e0] [node068:02208] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f75d7ccfa3f] [node068:02208] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node068:02208] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node068:02208] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node068:02208] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f75d767c76d] [node068:02208] [16] ./gibbs-gmm() [0x4aedc9] [node068:02208] *** End of error message *** [node067:02220] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node067:02220] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fe5b45a9a89] [node067:02220] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fe5b45aa7fb] [node067:02220] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fe5b45aa8e0] [node067:02220] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fe5b45aaa3f] [node067:02220] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node067:02220] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node067:02220] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node067:02220] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fe5b3f5776d] [node067:02220] [16] ./gibbs-gmm() [0x4aedc9] [node067:02220] *** End of error message *** [node069:02130] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f421091996e] [node069:02130] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node069:02130] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f4210902a89] [node069:02130] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f42109037fb] [node069:02130] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f42109038e0] [node069:02130] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f4210903a3f] [node069:02130] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node069:02130] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node069:02130] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node069:02130] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f42102b076d] [node069:02130] [16] ./gibbs-gmm() [0x4aedc9] [node069:02130] *** End of error message *** [node073:02213] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7ff386661cb0] [node073:02213] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7ff383b71425] [node073:02213] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7ff383b74b8b] [node073:02213] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node073:02213] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node073:02213] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node073:02213] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7ff383b5c76d] [node080:02351] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f16a1a8d873] [node080:02351] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f16a1a8d96e] [node080:02351] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node080:02351] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f16a1a76a89] [node080:02351] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f16a1a777fb] [node080:02351] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f16a1a778e0] [node080:02351] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f16a1a77a3f] [node080:02351] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node080:02351] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node080:02351] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node080:02351] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f16a142476d] [node080:02351] [16] ./gibbs-gmm() [0x4aedc9] [node070:02126] *** Process received signal *** [node070:02126] Signal: Aborted (6) [node070:02126] Signal code: (-6) terminate called after throwing an instance of 'std::bad_alloc' [node077:02219] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f7b00cf7a89] [node077:02219] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f7b00cf87fb] [node077:02219] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f7b00cf88e0] [node077:02219] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f7b00cf8a3f] [node077:02219] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node077:02219] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node077:02219] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node077:02219] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f7b006a576d] [node077:02219] [16] ./gibbs-gmm() [0x4aedc9] [node077:02219] *** End of error message *** [node071:02193] *** End of error message *** [node084:02188] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fa88890796e] [node084:02188] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node084:02188] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fa8888f0a89] [node084:02188] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fa8888f17fb] [node084:02188] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fa8888f18e0] [node084:02188] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fa8888f1a3f] [node084:02188] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node084:02188] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node084:02188] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node084:02188] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa88829e76d] [node084:02188] [16] ./gibbs-gmm() [0x4aedc9] [node084:02188] *** End of error message *** [node085:02135] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f77f658696e] [node085:02135] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node085:02135] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f77f656fa89] [node085:02135] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f77f65707fb] [node085:02135] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f77f65708e0] [node085:02135] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f77f6570a3f] [node085:02135] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node085:02135] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node085:02135] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node085:02135] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f77f5f1d76d] [node085:02135] [16] ./gibbs-gmm() [0x4aedc9] [node085:02135] *** End of error message *** [node009:02369] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f929330176d] [node009:02369] [ 7] ./gibbs-gmm() [0x4aedc9] [node009:02369] *** End of error message *** what(): std::bad_alloc what(): std::bad_alloc ERROR: basic_types.hpp(exec:140): Check failed: !(iarc.fail()) [node073:02213] [ 7] ./gibbs-gmm() [0x4aedc9] [node073:02213] *** End of error message *** [node080:02351] *** End of error message *** [node070:02126] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fbaa85ffcb0] [node070:02126] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fbaa5b0f425] [node070:02126] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fbaa5b12b8b] [node070:02126] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node070:02126] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node070:02126] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node070:02126] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fbaa5afa76d] [node070:02126] [ 7] ./gibbs-gmm() [0x4aedc9] [node070:02126] *** End of error message *** what(): std::bad_alloc [node018:02428] *** Process received signal *** [node014:02387] *** Process received signal *** [node054:02188] *** Process received signal *** [node078:02129] *** Process received signal *** [node018:02428] Signal: Aborted (6) [node018:02428] Signal code: (-6) [node014:02387] Signal: Aborted (6) [node014:02387] Signal code: (-6) [node054:02188] Signal: Aborted (6) [node054:02188] Signal code: (-6) [node078:02129] Signal: Aborted (6) [node078:02129] Signal code: (-6) [node018:02428] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fd17ad96cb0] [node018:02428] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fd1782a6425] [node018:02428] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fd1782a9b8b] [node018:02428] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7fd1788fc69d] [node018:02428] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7fd1788fa846] [node018:02428] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7fd1788fa873] [node018:02428] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7fd1788fa96e] [node014:02387] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f03f08e5cb0] [node014:02387] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f03eddf5425] [node014:02387] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f03eddf8b8b] [node014:02387] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f03ee44b69d] [node014:02387] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f03ee449846] [node014:02387] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f03ee449873] [node014:02387] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f03ee44996e] [node054:02188] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fc1ab3ddcb0] [node054:02188] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fc1a88ed425] [node054:02188] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fc1a88f0b8b] [node054:02188] [ 3] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0x15d3) [0x5b6053] [node054:02188] [ 4] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node054:02188] [ 5] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node078:02129] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f2d6e85acb0] [node078:02129] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f2d6bd6a425] [node078:02129] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f2d6bd6db8b] [node078:02129] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x11d) [0x7f2d6c3c069d] [node078:02129] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5846) [0x7f2d6c3be846] [node078:02129] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb5873) [0x7f2d6c3be873] [node018:02428] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node018:02428] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7fd1788e3a89] [node018:02428] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7fd1788e47fb] [node018:02428] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7fd1788e48e0] [node018:02428] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7fd1788e4a3f] [node018:02428] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node018:02428] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node018:02428] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node018:02428] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fd17829176d] [node018:02428] [16] ./gibbs-gmm() [0x4aedc9] [node018:02428] *** End of error message *** [node014:02387] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node014:02387] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f03ee432a89] [node014:02387] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f03ee4337fb] [node014:02387] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f03ee4338e0] [node014:02387] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f03ee433a3f] [node014:02387] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node014:02387] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node014:02387] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node014:02387] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f03edde076d] [node014:02387] [16] ./gibbs-gmm() [0x4aedc9] [node014:02387] *** End of error message *** [node054:02188] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fc1a88d876d] [node054:02188] [ 7] ./gibbs-gmm() [0x4aedc9] [node054:02188] *** End of error message *** [node078:02129] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb596e) [0x7f2d6c3be96e] [node078:02129] [ 7] ./gibbs-gmm(tc_new+0x433) [0x65ba43] [node078:02129] [ 8] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59) [0x7f2d6c3a7a89] [node078:02129] [ 9] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep8_M_cloneERKSaIcEm+0x2b) [0x7f2d6c3a87fb] [node078:02129] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs7reserveEm+0x30) [0x7f2d6c3a88e0] [node078:02129] [11] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6appendEmc+0x7f) [0x7f2d6c3a8a3f] [node078:02129] [12] ./gibbs-gmm(_ZN8graphlab19init_param_from_mpiERNS_13dc_init_paramENS_12dc_comm_typeE+0xe80) [0x5b5900] [node078:02129] [13] ./gibbs-gmm(_ZN8graphlab19distributed_controlC1Ev+0x7c9) [0x5a9509] [node078:02129] [14] ./gibbs-gmm(main+0x5b7) [0x4a96f7] [node078:02129] [15] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f2d6bd5576d] [node078:02129] [16] ./gibbs-gmm() [0x4aedc9] [node078:02129] *** End of error message *** [node051][[15035,1],50][../../../../../../ompi/mca/btl/tcp/btl_tcp_frag.c:216:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) -------------------------------------------------------------------------- mpiexec noticed that process rank 51 with PID 2155 on node node052 exited on signal 6 (Aborted). -------------------------------------------------------------------------- [node048][[15035,1],47][../../../../../../ompi/mca/btl/tcp/btl_tcp_frag.c:216:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) [node044][[15035,1],43][../../../../../../ompi/mca/btl/tcp/btl_tcp_frag.c:216:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) [node050][[15035,1],49][../../../../../../ompi/mca/btl/tcp/btl_tcp_frag.c:216:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) [node050][[15035,1],49][../../../../../../ompi/mca/btl/tcp/btl_tcp_frag.c:216:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) [node065][[15035,1],64][../../../../../../ompi/mca/btl/tcp/btl_tcp_frag.c:216:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104)