It looks like this should work, given "bldr.obj()" in your working example calls decouple as well to transfer buffer ownership. Can you provide a stack trace for this? I will try to reproduce as well.
I was not able to reproduce this running through the decouple operation. Please provide stack trace and more details. Even better would be to share buildable/runnable sample code that reproduces. Thanks!
On Tuesday, November 13, 2012 3:24:21 PM UTC-5, James wrote:
> It looks like this should work, given "bldr.obj()" in your working example > calls decouple as well to transfer buffer ownership. Can you provide a > stack trace for this? I will try to reproduce as well.
> On Tuesday, November 13, 2012 12:28:28 PM UTC-5, khanalesb wrote:
>> I am trying to use BSONObjBuilder as a serializer and decouple to avoid >> memcopy.
On Tuesday, November 13, 2012 5:37:09 PM UTC-5, James wrote:
> I was not able to reproduce this running through the decouple operation. > Please provide stack trace and more details. Even better would be to share > buildable/runnable sample code that reproduces. Thanks!
> On Tuesday, November 13, 2012 3:24:21 PM UTC-5, James wrote:
>> It looks like this should work, given "bldr.obj()" in your working >> example calls decouple as well to transfer buffer ownership. Can you >> provide a stack trace for this? I will try to reproduce as well.
>> On Tuesday, November 13, 2012 12:28:28 PM UTC-5, khanalesb wrote:
>>> I am trying to use BSONObjBuilder as a serializer and decouple to avoid >>> memcopy.
>>> Do I have to call anything else before decouple? Can I use it at all? >>> Thanks.
This works: int len; char* buf = bldr.decouple(len); char* buf1 = (char*) malloc(len); memcpy(buf1, buf, len); zmq::message_t replyBody( buf1, len, zmqFree ); but it is malloc and memcpy... In original example: int len; char* buf = bldr.decouple(len); zmq::message_t replyBody( buf, len, zmqFree ); it cores when 0mq calls my clallback to free buffer(zmqFree). All zmqFree does is free... inline void zmqFree(void* ptr, void* hint) { ::free(ptr);
> Do I have to call anything else before decouple? Can I use it at all? > Thanks.
Stacktrace: (gdb) where #0 0x00007ffff638a8a5 in raise () from /lib64/libc.so.6 #1 0x00007ffff638c085 in abort () from /lib64/libc.so.6 #2 0x00007ffff63c7a37 in __libc_message () from /lib64/libc.so.6 #3 0x00007ffff63cd366 in malloc_printerr () from /lib64/libc.so.6 #4 0x0000000000413b40 in its::hvr::ps::zmqFree (ptr=0x7fffe4017314, hint=0x0) at ../src/CommonDefs.cpp:16 #5 0x00007ffff7b6b0f2 in zmq_msg_close (msg_=0x7fffec0013e0) at zmq.cpp:152 #6 zmq_msg_close (msg_=0x7fffec0013e0) at zmq.cpp:130 #7 0x00007ffff7b52fa5 in zmq::encoder_t::message_ready (this=0x7fffec001398) at encoder.cpp:56 #8 0x00007ffff7b6c914 in get_data (this=0x7fffec0012e0) at encoder.hpp:80 #9 zmq::zmq_engine_t::out_event (this=0x7fffec0012e0) at zmq_engine.cpp:165 #10 0x00007ffff7b5773e in zmq::object_t::process_command (this=0x7fffdc005d10, cmd_=...) at object.cpp:63 #11 0x00007ffff7b546ec in zmq::io_thread_t::in_event (this=0x7391d0) at io_thread.cpp:83 #12 0x00007ffff7b5373e in zmq::epoll_t::loop (this=0x7395c0) at epoll.cpp:161 #13 0x00007ffff7b67566 in thread_routine (arg_=0x739630) at thread.cpp:75 #14 0x00007ffff6142851 in start_thread () from /lib64/libpthread.so.0 #15 0x00007ffff643f67d in clone () from /lib64/libc.so.6
On Wednesday, November 14, 2012 10:55:47 AM UTC-5, khanalesb wrote:
> On Tuesday, November 13, 2012 5:37:09 PM UTC-5, James wrote:
>> I was not able to reproduce this running through the decouple operation. >> Please provide stack trace and more details. Even better would be to share >> buildable/runnable sample code that reproduces. Thanks!
>> On Tuesday, November 13, 2012 3:24:21 PM UTC-5, James wrote:
>>> It looks like this should work, given "bldr.obj()" in your working >>> example calls decouple as well to transfer buffer ownership. Can you >>> provide a stack trace for this? I will try to reproduce as well.
>>> On Tuesday, November 13, 2012 12:28:28 PM UTC-5, khanalesb wrote:
>>>> I am trying to use BSONObjBuilder as a serializer and decouple to avoid >>>> memcopy.
int len; char* buf = bldrReply.decouple(len); std::cerr << (void*) buf << "\n"; free(buf); This cores on free call!!!! Should I use different constructor?
I can reproduce now. Thanks for the clarification on crash location. The constructor you are using should be fine. Will debug and get back to you with my findings.
On Wednesday, November 14, 2012 11:29:21 AM UTC-5, khanalesb wrote:
> On Wednesday, November 14, 2012 10:55:47 AM UTC-5, khanalesb wrote:
>> On Tuesday, November 13, 2012 5:37:09 PM UTC-5, James wrote:
>>> I was not able to reproduce this running through the decouple operation. >>> Please provide stack trace and more details. Even better would be to share >>> buildable/runnable sample code that reproduces. Thanks!
>>> On Tuesday, November 13, 2012 3:24:21 PM UTC-5, James wrote:
>>>> It looks like this should work, given "bldr.obj()" in your working >>>> example calls decouple as well to transfer buffer ownership. Can you >>>> provide a stack trace for this? I will try to reproduce as well.
>>>> On Tuesday, November 13, 2012 12:28:28 PM UTC-5, khanalesb wrote:
>>>>> I am trying to use BSONObjBuilder as a serializer and decouple to >>>>> avoid memcopy.
> int len; > char* buf = bldrReply.decouple(len); > std::cerr << (void*) buf << "\n"; > free(buf); > This cores on free call!!!! > Should I use different constructor?
On Wednesday, November 14, 2012 11:41:47 AM UTC-5, James wrote:
> I can reproduce now. Thanks for the clarification on crash location. The > constructor you are using should be fine. Will debug and get back to you > with my findings.
> On Wednesday, November 14, 2012 11:29:21 AM UTC-5, khanalesb wrote:
>> On Wednesday, November 14, 2012 10:55:47 AM UTC-5, khanalesb wrote:
>>> On Tuesday, November 13, 2012 5:37:09 PM UTC-5, James wrote:
>>>> I was not able to reproduce this running through the decouple >>>> operation. Please provide stack trace and more details. Even better would >>>> be to share buildable/runnable sample code that reproduces. Thanks!
>>>> On Tuesday, November 13, 2012 3:24:21 PM UTC-5, James wrote:
>>>>> It looks like this should work, given "bldr.obj()" in your working >>>>> example calls decouple as well to transfer buffer ownership. Can you >>>>> provide a stack trace for this? I will try to reproduce as well.
>>>>> On Tuesday, November 13, 2012 12:28:28 PM UTC-5, khanalesb wrote:
>>>>>> I am trying to use BSONObjBuilder as a serializer and decouple to >>>>>> avoid memcopy.
>> int len; >> char* buf = bldrReply.decouple(len); >> std::cerr << (void*) buf << "\n"; >> free(buf); >> This cores on free call!!!! >> Should I use different constructor?
decouple returns buffer + offset rather than just buffer !!!! !!! char *data = _b.buf() + _offset; !!! How do I get buffer to free ?????? char* _done() { if ( _doneCalled ) return _b.buf() + _offset;
Your original usage of the decouple method was correct. This is definitely a bug in the implementation of BSONObjBuilder. A ticket has been entered to fix: https://jira.mongodb.org/browse/SERVER-7671
While waiting on a fix my suggestion would to use your original implementation (with the memcpy) if you can afford to do so or test an alternate solution very carefully. Please track the ticket above for status on the fix.
On Thursday, November 15, 2012 12:29:51 PM UTC-5, James wrote:
> Your original usage of the decouple method was correct. This is definitely > a bug in the implementation of BSONObjBuilder. A ticket has been entered to > fix: > https://jira.mongodb.org/browse/SERVER-7671
> While waiting on a fix my suggestion would to use your original > implementation (with the memcpy) if you can afford to do so or test an > alternate solution very carefully. Please track the ticket above for status > on the fix.
> Thanks for your help in identifying this!
> James
> On Wednesday, November 14, 2012 12:26:44 PM UTC-5, khanalesb wrote:
>> On Tuesday, November 13, 2012 12:28:28 PM UTC-5, khanalesb wrote:
>>> I am trying to use BSONObjBuilder as a serializer and decouple to avoid >>> memcopy.
>>> Do I have to call anything else before decouple? Can I use it at all? >>> Thanks.
>> Working version: >> mongo::BufBuilder& bb = bldrReply.bb(); >> void* hint = bb.buf(); >> int len; >> char* buf = bldrReply.decouple(len); >> zmq::message_t reply( buf, len, zmqFreeHint, hint );
>> Actually call free on buffer from builder - ugly. Better idea?
Hmm... If you do fix - every user is broken. On the other hand in doc/code it says that has to be freed (::free()) Do you expect to change BufBuilder? Reason not to keep this implementation?
