MergeFrom/CopyFrom crashing

[Orange]

Greetings,

I'm having an issue where I'm causing MergeFrom() and CopyFrom() to
crash. I'm wondering if I'm doing something idiotic =).

I have something similar to this in my .proto files:
message Foo {
optional int32 msg_type = 1;
optional Bar bar = 17;
}

message Bar {
// about 17k of nested messages and plain data types
}

I'm trying to use Foo as a an encapsulating message frame, which will
be able to hold messages of type Bar, and other things in the future.

In my C++ code I have:

// Code to build up Bar my_bar_
....
Bar * foo_bar = my_foo_.mutable_bar(); // my_foo_ is a member
variable
printf("Calling MergeFrom\n");
foo_bar->MergeFrom(my_bar_); // This never returns?
printf("Done!\n");

I tried using CopyFrom instead of MergeFrom, and had the same outcome.
Am I using these methods correctly? Is there a different way I should
be assigning my_bar_ into my encapsulating my_foo_?

Thanks!

[Kenton Varda]

It sounds like your "Bar" type is a lot larger than protocol buffers are intended to be, but it should still work.  Can you run your code in a debugger and see where it is getting stuck?

Also, does using optimize_for = SPEED make a difference?

[The Orange Rider]

Using optimize_for = SPEED worked like a charm!

I'll try to look into why the other case is crashing. Oddly enough it's been awhile since I've used gdb =)

I'll update my findings when they're found. Thanks again.

[Jon Skeet]

On Jul 18, 10:35 pm, "The Orange Rider" <theorangeri...@gmail.com>
wrote:

> Using optimize_for = SPEED worked like a charm!
>
> I'll try to look into why the other case is crashing. Oddly enough it's been
> awhile since I've used gdb =)
>
> I'll update my findings when they're found. Thanks again.

Out of interest, are you able to test the Java version? I'd be
interested to hear whether or not that works (both in speed and size
versions).

Jon

[Orange]

> Out of interest, are you able to test the Java version? I'd be
> interested to hear whether or not that works (both in speed and size
> versions).

Unfortunately, no. My project involves Python and C++ only. I am
actually having problems with my Python code correctly parsing the
messages from C++, and it seems to be partially due to the large data
size in of my messages, but I think I'm doing something else wrong on
my end.

[Orange]

Well, I have a backtrace for when it's not optimized for speed:

#0 0xb72724d9 in google::protobuf::Message::MergeFrom
(this=0x8139c78,
from=@0x814ef50) at google/protobuf/message.cc:60
#1 0xb7274a87 in google::protobuf::internal::ReflectionOps::Merge (
descriptor=0x81520c8, from=@0x81b9398, to=0x81ba1e4)
at google/protobuf/reflection_ops.cc:94
#2 0xb72725bb in google::protobuf::Message::MergeFrom
(this=0x81ba1e0,
from=@0x81b9394) at google/protobuf/message.cc:61

It looks like the message.GetDescriptor() call is failing. I'm not
sure why yet =). Also, although MergeFrom() isn't crashing on me when
I use optimize_for = SPEED, it doesn't seem to work once I introduce
1795 bytes of data inside my message =/.

I'll keep you posted.

[Orange]

I figured out my problem. I was inadvertently corrupting one of my
nested messages (I had assigned my outdated non-protocol buffers
struct to its new protocol buffer struct equivalent). So it wasn't a
size issue so much me making messages too complex for my own good.
This caused things to crash when calling MergeFrom(), as the data was
invalid.

Thanks for the help, and I hope this helps others who do something
similar =)