Segmentation fault while deserializing blob data

[kua...@gmail.com]

Hello,

I am new to capnproto and may be doing something silly. I am trying to create a Data field of size 1526144 and it serializes without any issue. But when I try to deserialize the same message it fails with segfault.

If I decrease the Data size to about 8K it works without any problem. I could not find a documentation that explains this limit.

I am using following versions:

capnp --version

Cap'n Proto version 0.6.1

gcc --version

gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0

uname -r

4.15.0-74-generic

OS: Ubuntu 18.04.3 LTS

Message definition:

struct FrameData {

  frameId @0 :UInt32;

  image @1 :Data;

}

Code:

        capnp::MallocMessageBuilder msg_builder;

        FrameData::Builder frame = msg_builder.initRoot<FrameData>();

        frame.setFrameId(1);

        int size = 1526144;

        uint8_t* addr = (uint8_t*) malloc(size);

        memset(addr, 0, size);

        auto data = capnp::Data::Reader((kj::byte*)addr, size);

        frame.setImage(kj::arrayPtr(addr, size));

        auto words = capnp::messageToFlatArray(msg_builder);

        capnp::FlatArrayMessageReader msg2(words.asPtr()); 

        FrameData::Reader frame2 = msg2.getRoot<FrameData>();

        std::cout << frame2.getFrameId() << std::endl;

Running under GDB:

Thread 3 "libavtests" received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7fffe0669700 (LWP 8720)]

0x00005555555bf03e in kj::ArrayDisposer::Dispose_<kj::ArrayPtr<capnp::word const>, true>::dispose (firstElement=0x55555586b0e0 <kj::_::HeapArrayDisposer::instance>, 

    elementCount=140736911018432, capacity=140736911018432, disposer=...) at /usr/local/include/kj/array.h:622

622         disposer.disposeImpl(const_cast<RemoveConst<T>*>(firstElement),

(gdb) bt

Thread 3 "libavtests" received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7fffdbfff700 (LWP 8138)]

0x00005555555bf0fe in kj::ArrayDisposer::Dispose_<kj::ArrayPtr<capnp::word const>, true>::dispose (firstElement=0x55555586c0e0 <kj::_::HeapArrayDisposer::instance>, 

    elementCount=140736954317192, capacity=140736954317192, disposer=...) at /usr/local/include/kj/array.h:622

622         disposer.disposeImpl(const_cast<RemoveConst<T>*>(firstElement),

(gdb) bt

#0  0x00005555555bf0fe in kj::ArrayDisposer::Dispose_<kj::ArrayPtr<capnp::word const>, true>::dispose(kj::ArrayPtr<capnp::word const>*, unsigned long, unsigned long, kj::ArrayDisposer const&) (firstElement=0x55555586c0e0 <kj::_::HeapArrayDisposer::instance>, elementCount=140736954317192, capacity=140736954317192, disposer=...)

    at /usr/local/include/kj/array.h:622

#1  0x00005555555bf0b8 in kj::ArrayDisposer::dispose<kj::ArrayPtr<capnp::word const> >(kj::ArrayPtr<capnp::word const>*, unsigned long, unsigned long) const

    (this=0x555555894908, firstElement=0x55555586c0e0 <kj::_::HeapArrayDisposer::instance>, elementCount=140736954317192, capacity=140736954317192)

    at /usr/local/include/kj/array.h:640

#2  0x00005555555bf085 in kj::Array<kj::ArrayPtr<capnp::word const> >::dispose() (this=0x7fffdbffea48) at /usr/local/include/kj/array.h:254

--Type <RET> for more, q to quit, c to continue without paging--

#3  0x00005555555bf024 in kj::Array<kj::ArrayPtr<capnp::word const> >::~Array() (this=0x7fffdbffea48, __in_chrg=<optimized out>) at /usr/local/include/kj/array.h:149

#4  0x00005555555befaa in capnp::FlatArrayMessageReader::~FlatArrayMessageReader() (this=0x7fffdbffe980, __in_chrg=<optimized out>) at /usr/include/capnp/serialize.h:53

#5  0x00005555555cab8d in server() () at /home/ks/workspace/capnptest/test.cpp:122

#6  0x00005555555cb4a0 in std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) (__f=@0x555555894908: 0x5555555ca80a <server()>)

    at /usr/include/c++/7/bits/invoke.h:60

#7  0x00005555555cb231 in std::__invoke<void (*)()>(void (*&&)()) (__fn=@0x555555894908: 0x5555555ca80a <server()>) at /usr/include/c++/7/bits/invoke.h:95

#8  0x00005555555cb83c in std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x555555894908) at /usr/include/c++/7/thread:234

#9  0x00005555555cb80d in std::thread::_Invoker<std::tuple<void (*)()> >::operator()() (this=0x555555894908) at /usr/include/c++/7/thread:243

#10 0x00005555555cb7ec in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() (this=0x555555894900) at /usr/include/c++/7/thread:186

#11 0x00007ffff7f34408 in std::execute_native_thread_routine(void*) (__p=0x555555894900)

    at /opt/conda/conda-bld/compilers_linux-64_1534514838838/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:80

#12 0x00007ffff5db86db in start_thread (arg=0x7fffdbfff700) at pthread_create.c:463

#13 0x00007ffff450c88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Please let me know if the code looks OK or if I have to change anything. 

[Kenton Varda]

Hmm, I don't see any problems in the code. The crash seems to be in the destructor for FlatArrayMessageBuilder, when freeing an array that was allocated in the constructor. I can't really see how this could happen unless the FlatArrayMessageBuilder's own memory was corrupted somehow.

Would you be able to write a complete self-contained test case which I can actually compile and run myself? (You've almost provided that already but I want to make sure I'm looking at exactly the same code end-to-end.)

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/8ff65d7a-c08b-43be-90cf-4098a07a6265%40googlegroups.com.

[Maruf Maniruzzaman]

Thanks Kenton! After posting I was trying a few things and among those was upgrading to version 0.7.0 which seems fixes the issue. I was trying to use version 0.6.1 because it was used in other part of the project and including the header files capnp was complaining about version mismatch. 

It is proving to be difficult reproducing the issue with a standalone test/ project (I have tried as in here https://github.com/kuasha/test-capnp) even with 0.6.1. I am assuming it is probably combination of different versions of the library on same machine and how it was being used with zeromq and some other libraries. Version 0.7.0 seems to be better for cmake build as well and I have just "recompiled" those other schema files (fortunately I have control over that part as well) and resolve the version mismatch problem.

In future before reporting an issue I'll make sure it can be proven with a test case separate from larger project :).

[Kenton Varda]

Hi Maruf,

I don't think much changed in that code between 0.6 and 0.7. It sounds like you might have some sort of memory corruption happening in your other code. I suggest running under valgrind and maybe some other sanitizers.

-Kenton

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/4dedc118-c34c-4b0d-9919-c4a0e2a6b94d%40googlegroups.com.