Issue 179 in protobuf: Visual C++ error C1091 when compiling protoc generated code with over 64k descriptor

[prot...@googlecode.com]

Status: New
Owner: ken...@google.com
Labels: Type-Defect Priority-Medium

New issue 179 by iiro.hietala: Visual C++ error C1091 when compiling protoc
generated code with over 64k descriptor
http://code.google.com/p/protobuf/issues/detail?id=179

What steps will reproduce the problem?

1. Extended EnumValueOptions with a custom string as follows:

extend google.protobuf.EnumValueOptions {
optional string message = 50000;
}

2. Use the string option in multiple enumeration values in single
enumeration proto like this:

enum ExampleEnum
{
ENUMERATION_OPTION_1 = 1 [(message) = "Enumeration option 1"];
ENUMERATION_OPTION_2 = 2 [(message) = "Enumeration option 2"];
ENUMERATION_OPTION_3 = 3 [(message) = "Enumeration option 3"];
ENUMERATION_OPTION_4 = 4 [(message) = "Enumeration option 4"];
...
ENUMERATION_OPTION_9999 = 9999 [(message) = "Enumeration option 9999"];
}

3. Generate the C++ sources from .proto and then compile the generated
code.

NOTE: The attached file contains the example VS2008 project.

What is the expected output? What do you see instead?

When compiling the libraries that use the protoc generated sources the
Visual C++ compiler will give the following error:

"fatal error C1091: compiler limit: string exceeds 65535 bytes in length"

Expected: The protoc generated source code compiles

Actual: The code will not compile due to C1091 error.

What version of the product are you using? On what operating system?

Protocol Buffers 2.3.0, Visual Studio 2005, Visual Studio 2008, Windows XP
Professional.

Please provide any additional information below:

Also attempted this on Linux environment. With GCC the project compiles
successfully.

Attachments:
prototest.zip 5.7 MB

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings

[prot...@googlecode.com]

Updates:
Status: Accepted

Comment #1 on issue 179 by ken...@google.com: Visual C++ error C1091 when

compiling protoc generated code with over 64k descriptor
http://code.google.com/p/protobuf/issues/detail?id=179

(No comment was entered for this change.)

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings

--

You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[prot...@googlecode.com]

Updates:
Status: New

Comment #2 on issue 179 by ken...@google.com: Visual C++ error C1091 when

compiling protoc generated code with over 64k descriptor
http://code.google.com/p/protobuf/issues/detail?id=179

This proves complicated to fix, because we have lots of code that passes
around this blob as a pointer and a size. We'd have to update all that
code to pass around a pointer to a more complex data structure which might
contain multiple segments. Any fix would have to be used with all
compilers, even ones which don't have this limitation.

Your example is a 10MB source file. I don't think we care to support 10MB
source files. Realistically, if a file is so big that its descriptor is
over 64k, the best solution may be to simply split up the file. If you are
stuffing an excessive amount of information in custom options, it may be
time to consider a different format for that information.

I'd like to see more demand before we put effort into fixing this.

[prot...@googlecode.com]

Comment #3 on issue 179 by iiro.hie...@gmail.com: Visual C++ error C1091

when compiling protoc generated code with over 64k descriptor
http://code.google.com/p/protobuf/issues/detail?id=179

OK, thanks! It might be a good idea to write down this limitation into
documentation.

[prot...@googlecode.com]

Comment #5 on issue 179 by orlov...@gmail.com: Visual C++ error C1091 when

compiling protoc generated code with over 64k descriptor
http://code.google.com/p/protobuf/issues/detail?id=179

Hi guys,
To avoid this limitation you have to create descriptor string in heap.
Something like:
{code}
#include <sstream>

std::ostringstream oss;

oss << "myString1" << "myString2" << "myString3" << "myString4";
{code}

oss.str() would now return an instance of the STL's std:: string class, and
oss.str().c_str() would return a const char*.

I have a patch to solve this issue in C++ protoc output.
Please find attached file.

Attachments:
cpp_file.cc 21.1 KB

[prot...@googlecode.com]

Comment #6 on issue 179 by AleL...@gmail.com: Visual C++ error C1091 when

compiling protoc generated code with over 64k descriptor
http://code.google.com/p/protobuf/issues/detail?id=179

Valera,

"static std::string s(oss.str());\n" is not an elegant solution at all. I
did a similar hack with just std::string :
std::string s;
s += "bla-bla-bla".

The problem with this approach is that std::string parses literal string
till '\0' symbol. And, yes, literal string can have '\0' symbols in the
middle.

For instance, if .proto definition has an enumeration with elements equals
to 0 it will be presented by protobuf as '\0' element in proto definition
string.

A bit more elegant solution based on std::vector<char> is attached.

Attachments:
cpp_file.cc 22.5 KB

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[prot...@googlecode.com]

Updates:
Status: WontFix

Comment #7 on issue 179 by xiaof...@google.com: Visual C++ error C1091 when

compiling protoc generated code with over 64k descriptor

https://code.google.com/p/protobuf/issues/detail?id=179

(No comment was entered for this change.)

[mayun...@gmail.com]

I saw it on google group.

Please send me the fix file please, Thanks!