Odd problem with protobuf implementation in C++

[G.]

Hi all,

I am using protobuf 2.4.1, and I encountered a weird issue:

I created the following .proto file:


message Auction {
// Bid request id
required bytes Id = 1;
optional bytes Ip = 2;
required int32 adId = 3;
required int32 adHeight = 4;
required int32 adWidth = 5;
optional string domain = 6;
optional string country = 7;
optional string region = 8;
required string exchangeUserId = 9;
optional string pageUrl = 10;
optional int32 publisherId = 11;
optional int32 timezoneOffset = 12;
optional string userAgent = 13;
required string identifier = 14;
}


message Bid {
// Bid request Id
required bytes Id = 1;
required int32 processingTime = 2;
required int32 adId = 3;
required float bid = 4;
required int32 advertiserId = 5;
required string creativeId = 6;
required string billableSeat = 7;
required int32 category = 8;
required int32 vendorType = 9;
required int32 strategyId = 10;
required string clickthroughUrl = 11;
required string HtmlSnippet = 12;
}

It compiles fine with protoc.exe.

However, when I tried assigning the fields, I noticed the following
phenomenon: the fields id, billableseat and htmlsnippet in Bid
structure share the same address! When one is assigned, so are the
other two.

What am I doing wrong? Has anyone encountered such a thing before?

Thanks,

G.

[Jeremiah Jordan]

How are you setting the data?
You should be using something like:
bid.set_HtmlSnippet("Stuff");
and
std::string html = bid.HtmlSnippet();

See: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message

[G.]

Hi,

Thanks for responding. This is exactly the way I access the fields.

Code snippet:
...
b.set_billableseat("Mediamind");
b.set_category(5);
b.set_clickthroughurl("http://www.ynet.co.il");
b.set_creativeid("ab15a");
...
I walked through this with the debugger. When billableseat is set,
Clickthroughurl and creativeId both become "Mediamind" as well. Same
with clickthroughUrl and creativeId - setting each one of them turns
all 3 fields the same value. Only setting Category didn't affect
anything except category.

Thanks,

G.

On Apr 4, 7:45 pm, Jeremiah Jordan <jeremiah.jor...@gmail.com> wrote:
> How are you setting the data?
> You should be using something like:
> bid.set_HtmlSnippet("Stuff");
> and
> std::string html = bid.HtmlSnippet();
>

> See:https://developers.google.com/protocol-buffers/docs/reference/cpp/goo...

[Henner Zeller]

On Thu, Apr 5, 2012 at 01:03, G. <geula.v...@mediamind.com> wrot> Hi,

>
> Thanks for responding. This is exactly the way I access the fields.
>
> Code snippet:
> ...
>        b.set_billableseat("Mediamind");
>        b.set_category(5);
>        b.set_clickthroughurl("http://www.ynet.co.il");
>        b.set_creativeid("ab15a");
> ...
> I walked through this with the debugger. When billableseat is set,
> Clickthroughurl and creativeId both become "Mediamind" as well. Same
> with clickthroughUrl and creativeId - setting each one of them turns
> all 3 fields the same value. Only setting Category didn't affect
> anything except category.

What compiler are you using ?

Protocol buffers definitely don't behave this way, so only a very
broken compiler or something peculiar in your memory management or
other setup can cause this.

(IIRC, default values in protobufs share a const instance (in this
case: the empty string) for memory reasons, so you would see the same
address for the default values, but this is of course disengaged
whenever a value is set.)

If you just create a simple protocol buffer with two string fields and
a simple main() function: do you get the same behavior ?

-h

> --
> You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
> To post to this group, send email to prot...@googlegroups.com.
> To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
>

[Christopher Smith]

This sounds like something *very* weird is going wrong in your runtime. What does the debugger show as the memory locations of the string fields? the memory locations the allocator provided for the string data?

I think if you make a stand alone unit test for this, you won't see this behaviour.

--Chris

[G.]

Hi Hans,

In my proto file, there are two structures, Auction and Bid. The
Auction structure behaves normally - setting the fields works
correctly. I am also using the pre-defined realtime-bidding.proto
file, all of which structures are working fine. It is only the Bid
structure that acts broken, and I can't see why - because there is
nothing special about its definition.

I am using the protoc.exe compiler that came with the protobuf 2.4.1
installation package for windows, and my IDE is visual studio 2010
professional, everything is compiled in x64.

G.

On Apr 5, 7:54 pm, Henner Zeller <henner.zel...@googlemail.com> wrote:
> On Thu, Apr 5, 2012 at 01:03, G. <geula.vainap...@mediamind.com> wrot> Hi,

[Henner Zeller]

On Sat, Apr 7, 2012 at 21:18, G. <geula.v...@mediamind.com> wrote:
> Hi Hans,
>
> In my proto file, there are two structures, Auction and Bid. The
> Auction structure behaves normally - setting the fields works
> correctly. I am also using the pre-defined realtime-bidding.proto
> file, all of which structures are working fine. It is only the Bid
> structure that acts broken, and I can't see why - because there is
> nothing special about its definition.
>
> I am using the protoc.exe compiler that came with the protobuf 2.4.1
> installation package for windows, and my IDE is visual studio 2010
> professional, everything is compiled in x64.

One idea I can think of is that you forgot to allocate the Bid
structure and your compiler sets unallocated pointers to some default
value. Or Bid is already freed (a common technique in memory
allocators is to fill the deleted value with some pattern in debug
mode to detect this early). Setting values in your case then
accidentally does not crash but modifies the data structure the
garbage pointer happens to point to.

How do you get the instance of Bid ? It is a bit hard to debug your
code without seeing a self-contained snippet that exposes the problem.
In your example, you're using b.set_.... Is 'b' a reference in this
case ? Could it be that you set values on a reference to a temporary
that already is gone ?
Try to run your program with valgrind maybe to find a broken allocation.

(Note it is very very unlikely that the protobuf C++ implementation
has such a bug that is undetected, so something else in your program
is going wrong.)

-h

[G.]

Problem was resolved, in a totally unrelated manner...

It was decided to implement the part that was supposed to set the Bid
structure, in C#. The problem never recurred there. So I guess I will
never know what was wrong.

Thanks to all who answered!

G.

[Thomas ASTIER]

Hi!

After a small research, I ended up on this group, that discussed the exact problem I am currently experiencing.

Unfortunately, the solution could not be applied to my case.

By posting on this old thread, I am hoping somebody could give me some more hints in order to resolve my situation.

Please, find bellow the message I posted on another dev forum:

 

Currently using protobuf under its 2.5.0 version, I am stuck with a problem I am struggling to resolve. First of all, the following .proto files represent the messages I am dealing with :

MacroMission.proto
message MacroMission{
    required int32 timeWithoutCom=1;
    required string name=2;
    required Phase failSafePhase=3;
    repeated Phase phases=4;
}
message Phase{
    optional Survey survey=1;
}
message Survey{
    repeated Segment trajectory=1;
        required string sonarPreset=4;
}
message Segment{
        required WayPoint start=1;
        required WayPoint end=2;
}
Common.proto
message WayPoint
{
   required GeoPoint pos = 1;   // Not relevant
   optional string label = 2 ;
}

The aim of the software is to fill a MacroMission message. I am working under Windows 7, with C++/Qt and minGW compiler, using the protobuf library within Qt Plugin projects (a message is shared between two distinct plugins, a « creator » and a « filler »).

My problem is : whenever I try to assign a value to a field of string type, all the other string fields are taking the same value. My first guess was that, somehow, all these fields had the same address in memory, which turned out true. Concerned about the good initialization of the messages, I double checked all the process for creating the MacroMission. It could be best described by the following lines :

Classe A – Plugin 1
        ::pb::MacroMission mission;

        mission.set_name(rMissionName.toStdString());


        mission.set_timewithoutcom(timeWithoutCom);


        foreach(MissionClass)
        {
            ::pb::Phase * missionPhase = mission.add_phases();
            MissionClass->FillPhase(missionPhase);
        }

MissionClass – Plugin 2
    FillPhase methode (TnamedSegment is a support class for segment data access)

    ::pb::Survey * survey = missionPhase->mutable_survey();  

    for(int segmentRank = 0 ; segmentRank < surveyParams.getSegments().count() ; ++segmentRank)
    {
        TNamedSegment segment = surveyParams.getSegments()[segmentRank];

        qDebug() << "Adding one segment :";

        ::pb::Segment* pb_segment = survey->add_trajectory();

        pb_segment->mutable_start()->set_label(QString("WP_%1_start").arg(segmentRank).toStdString());
        pb_segment->mutable_start()->mutable_pos()->set_latitude(segment.startWP.latitude);
        pb_segment->mutable_start()->mutable_pos()->set_longitude(segment.startWP.longitude);

        pb_segment->mutable_end()->set_label(QString("WP_%1_end").arg(segmentRank).toStdString());
        pb_segment->mutable_end()->mutable_pos()->set_latitude(segment.endWP.latitude);
        pb_segment->mutable_end()->mutable_pos()->set_longitude(segment.endWP.longitude);

}

    survey->set_sonarpreset(surveyParams.getSonarPreset().toStdString());

In the end, nothing comes out of the blue in this code. But, as I previously said, all the protobuf labels/names in the messages have the same address. Therefore, they will all end up with the same value, the one of the last field set (in this case, the value of SonarPreset). I have been able to solve a little tiny part of the problem by using the set_allocated_var method, but in the end, the major part of the problem remains unchanged.

After several hours of internet researches, many regenerations/rebuilds of the protobuf messages and of the project in which they are used, memory allocation double checks, and protobuf library help documentation readings, I come to you in order to know if, by any change, someone would have the beginning of a solution to my problem.

I did a small unit test program which only fill the protobuff message in a single "main.cpp" file, and everything works fine...

Thank you in advance for any answer.

PS : Please let me know if you need any further information in order to clarify the problem.