value_.SetNoArena causes my process terminated with signal 6, Aborted.

[Boris Pitel]

I am running a process on RedHat 7 which uses intensively protobuf. Application is multithreaded and usually runs wihout problems for days. It processes billions of messages per day.

Once in a while the program gets aborted. My gdb on core tells that the abort was initiated in protobuf generated file. 

Here is bt output from gdb

#0  0x00007f4a72129377 in raise () from /lib64/libc.so.6

#1  0x00007f4a7212aa68 in abort () from /lib64/libc.so.6

#2  0x00007f4a7216bec7 in __libc_message () from /lib64/libc.so.6

#3  0x00007f4a721746b9 in _int_free () from /lib64/libc.so.6

#4  0x00007f4a72a96b63 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() () from /lib64/libstdc++.so.6

#5  0x00007f4a76eb4f72 in interfaces::MIObjectItemType::set_value (this=0x7f4a10244ba0, value=0x7f4a31cf5af0 "922")

    at /home/boris_pitel-ciqdev.com/transformer/TransformerSupport/../IncludesProject/MIObjectSetTypeProtos.pb.h:794

#6  0x00007f4a76eb5b34 in PriceTicMIObject::set_KeyExchangeSource (this=0x7f4a101d4bd0, v=0x7f4a31cf5af0 "922")

    at /home/boris_pitel-ciqdev.com/transformer/TransformerSupport/../IncludesProject/PriceTicObjectSet.h:300

#7  0x00007f4a76eb555d in PriceTicMIObject::set_KeyExchangeSource (this=0x7f4a101d4bd0, v=922)

    at /home/boris_pitel-ciqdev.com/transformer/TransformerSupport/../IncludesProject/PriceTicObjectSet.h:128

#

On this stack the file /MIObjectSetTypeProtos.pb.h is generated by protobuf

line 794 points to value_.SetNoArena in function below.

inline void MIObjectItemType::set_value(const char* value) {

  GOOGLE_DCHECK(value != NULL);

  

  value_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value));

  // @@protoc_insertion_point(field_set_char:interfaces.MIObjectItemType.value)

}

I realize that it can be some memory corruption on my side. 

But I still hope that may be somebody from protobuf experts can have enlightening ideas of how to solve this issue, or somebody remembers similar situation.

Looking forward to any help.

-Boris