Segfault when parsing data.

[Test Last]

Hi everyone

I ported a ODBC Driver that works in an RPC fashion from windows to linux.

The port was actually quite trivial since the code was already Cross OS Friendly.

The protobuf has never been an issue on Windows side. The protobuf package was V3.12.3 and was also installed by visual studio.

However when I download Protobuf C++ and compile them on both Ubuntu and CentOS. And compile my program I get strange errors ALWAYS resulting from "repeated_field.h. 
Sometimes the error is on line 1750 and other times its on 1744 but no matter what version I compile or what OS I use it just never seems to want to process anything that needs to use "repeated_field.h" . I will paste my backtrace and also my ldd of the driver. With the makefile I use to compile the protobufs.

This is a backtrace of a string of data that needs to be parsed. This always breaks.
The class its trying to parse into is -> ExecuteResponse : Please check the proto files.

This is the backtrace.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6369cc4 in google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<ColumnValue>::TypeHandler> (this=0x68e8d0, prototype=0x0)
   at /usr/local/include/google/protobuf/repeated_field.h:1744
1744      if (rep_ != NULL && current_size_ < rep_->allocated_size) {
(gdb) backtrace
#0  0x00007ffff6369cc4 in google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<ColumnValue>::TypeHandler> (this=0x68e8d0, prototype=0x0)
   at /usr/local/include/google/protobuf/repeated_field.h:1744
#1  0x00007ffff6366e39 in google::protobuf::RepeatedPtrField<ColumnValue>::Add (this=0x68e8d0) at /usr/local/include/google/protobuf/repeated_field.h:2195
#2  0x00007ffff63646fa in Row::_internal_add_value (this=0x68e8c0) at common.pb.h:5781
#3  0x00007ffff635697f in Row::_InternalParse (this=0x68e8c0,  
   ptr=0x67c5f0 "*\n\023\b\025\032\017Alex", '*' <repeats 11 times>, "\"\023\b\025\032\017Alex", '*' <repeats 11 times>, "\n,\n\024\b\025\032\020a***oum.br\"\024\b\025\032
\020a***com.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\234\001\n*\n\023\b\025\032\017Edua"
, '*' <repeats 11 times>, "\"\023\b\025\032\017Edua", '*' <repeats 11 times>, "\n<\n\034\b\025\032\030e*****"..., ctx=0x7fffffffce30) at common.pb.cc:3927
#4  0x00007ffff6367db6 in google::protobuf::internal::ParseContext::ParseMessage<Row> (this=0x7fffffffce30, msg=0x68e8c0,  
   ptr=0x67c5ef "\n*\n\023\b\025\032\017Alex", '*' <repeats 11 times>, "\"\023\b\025\032\017Alex", '*' <repeats 11 times>, "\n,\n\024\b\025\032\020a*com.br\"\024\b\025\0
32\020a***oom.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\234\001\n*\n\023\b\025\032\017Edu
a", '*' <repeats 11 times>, "\"\023\b\025\032\017Edua", '*' <repeats 11 times>, "\n<\n\034\b\025\032\030e****"...) at /usr/local/include/google/protobuf/parse_context.h:614
#5  0x00007ffff635596f in Frame::_InternalParse (this=0x68d830,  
   ptr=0x67c5ed "\214\001\n*\n\023\b\025\032\017Alex", '*' <repeats 11 times>, "\"\023\b\025\032\017Alex", '*' <repeats 11 times>, "\n,\n\024\b\025\032\020a*l.com.br\"\024
\b\025\032\020aom.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\234\001\n*\n\023\b\025\03
2\017Edua", '*' <repeats 11 times>, "\"\023\b\025\032\017Edua", '*' <repeats 11 times>, "\n<\n\034\b\025\032\030e**"..., ctx=0x7fffffffce30) at common.pb.cc:3688
#6  0x00007ffff63ac758 in google::protobuf::internal::ParseContext::ParseMessage<Frame> (this=0x7fffffffce30, msg=0x68d830,  
   ptr=0x67c5ea "\020\001\032\214\001\n*\n\023\b\025\032\017Alex", '*' <repeats 11 times>, "\"\023\b\025\032\017Alex", '*' <repeats 11 times>, "\n,\n\024\b\025\032\020a*l.
com.br\"\024\b\025\032\020a****l.com.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\234\001\n*\n\
023\b\025\032\017Edua", '*' <repeats 11 times>, "\"\023\b\025\032\017Edua", '*' <repeats 11 times>, "\n<\n\034\b\025\032\030"...)
   at /usr/local/include/google/protobuf/parse_context.h:614
#7  0x00007ffff6395e79 in ResultSetResponse::_InternalParse (this=0x674310,  
   ptr=0x67c5e8 "\215\006\020\001\032\214\001\n*\n\023\b\025\032\017Alex", '*' <repeats 11 times>, "\"\023\b\025\032\017Alex", '*' <repeats 11 times>, "\n,\n\024\b\025\032\020a*
**com.br\"\024\b\025\032\020a*om.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\234\0
01\n*\n\023\b\025\032\017Edua", '*' <repeats 11 times>, "\"\023\b\025\032\017Edua", '*' <repeats 11 times>, "\n<\n\034\b\025"..., ctx=0x7fffffffce30) at response.pb.cc:788
#8  0x00007ffff63aca34 in google::protobuf::internal::ParseContext::ParseMessage<ResultSetResponse> (this=0x7fffffffce30, msg=0x674310,  
   ptr=0x67c4a3 "\n\b0pOR9266\020B\030\001\"\263\002\nJ \001\060\001@\377\001J\bfullnameR\bfullname`\377\001z\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\
006STRING\030\025\nF\b\001 \001\060\001@\377\001J\005emailR\005email`\377\001z\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\006STRING\030\025\nJ\b\002 \001\
060\001@\377\001J\acountryR\acountry`\377\001z\005S"...) at /usr/local/include/google/protobuf/parse_context.h:614
#9  0x00007ffff6397670 in ExecuteResponse::_InternalParse (this=0x7fffffffd300,  
   ptr=0x67c4a1 "\205\t\n\b0pOR9266\020B\030\001\"\263\002\nJ \001\060\001@\377\001J\bfullnameR\bfullname`\377\001z\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\
f\022\006STRING\030\025\nF\b\001 \001\060\001\377\001J\005emailR\005email`\377\001z\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\006STRING\030\025\nJ\b\002
\001\060\001\377\001J\acountryR\acountry`\377\001z"..., ctx=0x7fffffffce30) at response.pb.cc:1135
#10 0x00007ffff55350a0 in google::protobuf::internal::MergeFromImpl<false> (input=..., msg=0x7fffffffd300, parse_flags=google::protobuf::MessageLite::kParse)
   at ./google/protobuf/parse_context.h:224
#11 0x00007ffff62ad4b7 in proto::helper::parseNonStream<ExecuteResponse> (inc=0x7fffffffd0f0, outClass=0x7fffffffd300) at transProto.h:32
#12 0x00007ffff62a88b6 in ConnectEngine::PrepareAndExecuteReq (this=0x671170, query="select * from VidDemo ", PERResponse=0x7fffffffd300, maxRows=9223372036854775807,  
   frameMaxSize=2147483647, _statementId=66) at ConnectEngine.cpp:639
#13 0x00007ffff62b462c in Consolidation::executeQuery (this=0x6710b0, query="select * from VidDemo ") at Consolidation.cpp:173
#14 0x00007ffff62f3f9e in Statement::requestNextPackOfResultSets (this=0x68bfe0, mutator=...) at statement.cpp:215

//THIS IS MY MAKEFILE

LDFLAGS = -L/usr/local/lib `pkg-config --libs protobuf grpc++`\

-Wl,--no-as-needed -lgrpc++_reflection -Wl,--as-needed\

-ldl

CXX = g++

CPPFLAGS += `pkg-config --cflags protobuf grpc`

CXXFLAGS += -std=c++14

DEBUG = -g

GRPC_CPP_PLUGIN = grpc_cpp_plugin

GRPC_CPP_PLUGIN_PATH ?= `which $(GRPC_CPP_PLUGIN)`

all: common request response common.o request.o response.o

common:

protoc -I=. --cpp_out=. ./common.proto

request:

protoc -I=. --cpp_out=. ./request.proto

response:

protoc -I=. --cpp_out=. ./response.proto

common.o: common.pb.cc

$(CXX) $(DEBUG) -fPIC -c $^ $(LDFLAGS) -o $@ $(CXXFLAGS)

request.o: request.pb.cc

$(CXX) $(DEBUG) -fPIC -c $^ $(LDFLAGS) -o $@ $(CXXFLAGS)

response.o: response.pb.cc

$(CXX) $(DEBUG) -fPIC -c $^ $(LDFLAGS) -o $@ $(CXXFLAGS)

clean:

rm -f *.o *.pb.cc *.pb.h client

//This is my 3 Proto files. Its part of the Calcite Avatica package.

//COMMON.PROTO

syntax = "proto3";

//option cc_generic_services = true;

option cc_generic_services = false;

//option csharp_namespace = "Calcite.Avatica";

// Details about a connection

message ConnectionProperties {

bool is_dirty = 1;

bool auto_commit = 2;

bool has_auto_commit = 7; // field is a Boolean, need to discern null and default value

bool read_only = 3;

bool has_read_only = 8; // field is a Boolean, need to discern null and default value

uint32 transaction_isolation = 4;

string catalog = 5;

string schema = 6;

}

// Statement handle

message StatementHandle {

string connection_id = 1;

uint32 id = 2;

Signature signature = 3;

}

// Results of preparing a statement

message Signature {

repeated ColumnMetaData columns = 1;

string sql = 2;

repeated AvaticaParameter parameters = 3;

CursorFactory cursor_factory = 4;

StatementType statementType = 5;

}

// Has to be consistent with Meta.StatementType

enum StatementType {

SELECT = 0;

INSERT = 1;

UPDATE = 2;

DELETE = 3;

UPSERT = 4;

MERGE = 5;

OTHER_DML = 6;

CREATE = 7;

DROP = 8;

ALTER = 9;

OTHER_DDL = 10;

CALL = 11;

}

message ColumnMetaData {

uint32 ordinal = 1;

bool auto_increment = 2;

bool case_sensitive = 3;

bool searchable = 4;

bool currency = 5;

uint32 nullable = 6;

bool signed = 7;

uint32 display_size = 8;

string label = 9;

string column_name = 10;

string schema_name = 11;

uint32 precision = 12;

uint32 scale = 13;

string table_name = 14;

string catalog_name = 15;

bool read_only = 16;

bool writable = 17;

bool definitely_writable = 18;

string column_class_name = 19;

AvaticaType type = 20;

}

enum Rep {

PRIMITIVE_BOOLEAN = 0;

PRIMITIVE_BYTE = 1;

PRIMITIVE_CHAR = 2;

PRIMITIVE_SHORT = 3;

PRIMITIVE_INT = 4;

PRIMITIVE_LONG = 5;

PRIMITIVE_FLOAT = 6;

PRIMITIVE_DOUBLE = 7;

BOOLEAN = 8;

BYTE = 9;

CHARACTER = 10;

SHORT = 11;

INTEGER = 12;

LONG = 13;

FLOAT = 14;

DOUBLE = 15;

BIG_INTEGER = 25;

BIG_DECIMAL = 26;

JAVA_SQL_TIME = 16;

JAVA_SQL_TIMESTAMP = 17;

JAVA_SQL_DATE = 18;

JAVA_UTIL_DATE = 19;

BYTE_STRING = 20;

STRING = 21;

NUMBER = 22;

OBJECT = 23;

NULL = 24;

ARRAY = 27;

STRUCT = 28;

MULTISET = 29;

}

// Base class for a column type

message AvaticaType {

uint32 id = 1;

string name = 2;

Rep rep = 3;

repeated ColumnMetaData columns = 4; // Only present when name = STRUCT

AvaticaType component = 5; // Only present when name = ARRAY

}

// Metadata for a parameter

message AvaticaParameter {

bool signed = 1;

uint32 precision = 2;

uint32 scale = 3;

uint32 parameter_type = 4;

string type_name = 5;

string class_name = 6;

string name = 7;

}

// Information necessary to convert an Iterable into a Calcite Cursor

message CursorFactory {

enum Style {

OBJECT = 0;

RECORD = 1;

RECORD_PROJECTION = 2;

ARRAY = 3;

LIST = 4;

MAP = 5;

}

Style style = 1;

string class_name = 2;

repeated string field_names = 3;

}

// A collection of rows

message Frame {

uint64 offset = 1;

bool done = 2;

repeated Row rows = 3;

}

// A row is a collection of values

message Row {

repeated ColumnValue value = 1;

}

// Database property, list of functions the database provides for a certain operation

message DatabaseProperty {

string name = 1;

repeated string functions = 2;

}

// Message which encapsulates another message to support a single RPC endpoint

message WireMessage {

string name = 1;

bytes wrapped_message = 2;

}

// A value might be a TypedValue or an Array of TypedValue's

message ColumnValue {

repeated TypedValue value = 1; // deprecated, use array_value or scalar_value

repeated TypedValue array_value = 2;

bool has_array_value = 3; // Is an array value set?

TypedValue scalar_value = 4;

}

// Generic wrapper to support any SQL type. Struct-like to work around no polymorphism construct.

message TypedValue {

Rep type = 1; // The actual type that was serialized in the general attribute below

bool bool_value = 2; // boolean

string string_value = 3; // char/varchar

sint64 number_value = 4; // var-len encoding lets us shove anything from byte to long

// includes numeric types and date/time types.

bytes bytes_value = 5; // binary/varbinary

double double_value = 6; // big numbers

bool null = 7; // a null object

repeated TypedValue array_value = 8; // The Array

Rep component_type = 9; // If an Array, the representation for the array values

bool implicitly_null = 10; // Differentiate between explicitly null (user-set) and implicitly null

// (un-set by the user)

}

// The severity of some unexpected outcome to an operation.

// Protobuf enum values must be unique across all other enums

enum Severity {

UNKNOWN_SEVERITY = 0;

FATAL_SEVERITY = 1;

ERROR_SEVERITY = 2;

WARNING_SEVERITY = 3;

}

// Enumeration corresponding to DatabaseMetaData operations

enum MetaDataOperation {

GET_ATTRIBUTES = 0;

GET_BEST_ROW_IDENTIFIER = 1;

GET_CATALOGS = 2;

GET_CLIENT_INFO_PROPERTIES = 3;

GET_COLUMN_PRIVILEGES = 4;

GET_COLUMNS = 5;

GET_CROSS_REFERENCE = 6;

GET_EXPORTED_KEYS = 7;

GET_FUNCTION_COLUMNS = 8;

GET_FUNCTIONS = 9;

GET_IMPORTED_KEYS = 10;

GET_INDEX_INFO = 11;

GET_PRIMARY_KEYS = 12;

GET_PROCEDURE_COLUMNS = 13;

GET_PROCEDURES = 14;

GET_PSEUDO_COLUMNS = 15;

GET_SCHEMAS = 16;

GET_SCHEMAS_WITH_ARGS = 17;

GET_SUPER_TABLES = 18;

GET_SUPER_TYPES = 19;

GET_TABLE_PRIVILEGES = 20;

GET_TABLES = 21;

GET_TABLE_TYPES = 22;

GET_TYPE_INFO = 23;

GET_UDTS = 24;

GET_VERSION_COLUMNS = 25;

}

// Represents the breadth of arguments to DatabaseMetaData functions

message MetaDataOperationArgument {

enum ArgumentType {

STRING = 0;

BOOL = 1;

INT = 2;

REPEATED_STRING = 3;

REPEATED_INT = 4;

NULL = 5;

}

string string_value = 1;

bool bool_value = 2;

sint32 int_value = 3;

repeated string string_array_values = 4;

repeated sint32 int_array_values = 5;

ArgumentType type = 6;

}

enum StateType {

SQL = 0;

METADATA = 1;

}

message QueryState {

StateType type = 1;

string sql = 2;

MetaDataOperation op = 3;

repeated MetaDataOperationArgument args = 4;

bool has_args = 5;

bool has_sql = 6;

bool has_op = 7;

}

//REQUESTER.PROTO

syntax = "proto3";

option cc_generic_services = false;

//option java_package = "org.apache.calcite.avatica.proto";

import "common.proto";

// Request for Meta#getCatalogs()

message CatalogsRequest {

string connection_id = 1;

}

// Request for Meta#getDatabaseProperties()

message DatabasePropertyRequest {

string connection_id = 1;

}

// Request for Meta#getSchemas(String, org.apache.calcite.avatica.Meta.Pat)}

message SchemasRequest {

string catalog = 1;

string schema_pattern = 2;

string connection_id = 3;

}

// Request for Request for Meta#getTables(String, org.apache.calcite.avatica.Meta.Pat,

// org.apache.calcite.avatica.Meta.Pat, java.util.List)

message TablesRequest {

string catalog = 1;

string schema_pattern = 2;

string table_name_pattern = 3;

repeated string type_list = 4;

bool has_type_list = 6; // Having an empty type_list is distinct from a null type_list

string connection_id = 7;

}

// Request for Meta#getTableTypes()

message TableTypesRequest {

string connection_id = 1;

}

// Request for Meta#getColumns(String, org.apache.calcite.avatica.Meta.Pat,

// org.apache.calcite.avatica.Meta.Pat, org.apache.calcite.avatica.Meta.Pat).

message ColumnsRequest {

string catalog = 1;

string schema_pattern = 2;

string table_name_pattern = 3;

string column_name_pattern = 4;

string connection_id = 5;

}

// Request for Meta#getTypeInfo()

message TypeInfoRequest {

string connection_id = 1;

}

// Request for Meta#prepareAndExecute(Meta.StatementHandle, String, long, Meta.PrepareCallback)

message PrepareAndExecuteRequest {

string connection_id = 1;

string sql = 2;

uint64 max_row_count = 3; // Deprecated

uint32 statement_id = 4;

int64 max_rows_total = 5; // The maximum number of rows that will be allowed for this query

int32 first_frame_max_size = 6; // The maximum number of rows that will be returned in the

// first Frame returned for this query.

}

// Request for Meta.prepare(Meta.ConnectionHandle, String, long)

message PrepareRequest {

string connection_id = 1;

string sql = 2;

uint64 max_row_count = 3; // Deprecated

int64 max_rows_total = 4; // The maximum number of rows that will be allowed for this query

}

// Request for Meta#fetch(Meta.StatementHandle, List, long, int)

message FetchRequest {

string connection_id = 1;

uint32 statement_id = 2;

uint64 offset = 3;

uint32 fetch_max_row_count = 4; // Maximum number of rows to be returned in the frame. Negative means no limit. Deprecated!

int32 frame_max_size = 5;

}

// Request for Meta#createStatement(Meta.ConnectionHandle)

message CreateStatementRequest {

string connection_id = 1;

}

// Request for Meta#closeStatement(Meta.StatementHandle)

message CloseStatementRequest {

string connection_id = 1;

uint32 statement_id = 2;

}

// Request for Meta#openConnection(Meta.ConnectionHandle, Map<String, String>)

message OpenConnectionRequest {

string connection_id = 1;

map<string, string> info = 2;

}

// Request for Meta#closeConnection(Meta.ConnectionHandle)

message CloseConnectionRequest {

string connection_id = 1;

}

message ConnectionSyncRequest {

string connection_id = 1;

ConnectionProperties conn_props = 2;

}

// Request for Meta#execute(Meta.ConnectionHandle, list, long)

message ExecuteRequest {

StatementHandle statementHandle = 1;

repeated TypedValue parameter_values = 2;

uint64 deprecated_first_frame_max_size = 3; // Deprecated, use the signed int instead.

bool has_parameter_values = 4;

int32 first_frame_max_size = 5; // The maximum number of rows to return in the first Frame

}

message SyncResultsRequest {

string connection_id = 1;

uint32 statement_id = 2;

QueryState state = 3;

uint64 offset = 4;

}

// Request to invoke a commit on a Connection

message CommitRequest {

string connection_id = 1;

}

// Request to invoke rollback on a Connection

message RollbackRequest {

string connection_id = 1;

}

// Request to prepare and execute a collection of sql statements.

message PrepareAndExecuteBatchRequest {

string connection_id = 1;

uint32 statement_id = 2;

repeated string sql_commands = 3;

}

// Each command is a list of TypedValues

message UpdateBatch {

repeated TypedValue parameter_values = 1;

}

message ExecuteBatchRequest {

string connection_id = 1;

uint32 statement_id = 2;

repeated UpdateBatch updates = 3; // A batch of updates is a list<list<typevalue>>

}

//RESPONSE.PROTO

syntax = "proto3";

//option java_package = "org.apache.calcite.avatica.proto";

import "common.proto";

// Response that contains a result set.

message ResultSetResponse {

string connection_id = 1;

uint32 statement_id = 2;

bool own_statement = 3;

Signature signature = 4;

Frame first_frame = 5;

uint64 update_count = 6; // -1 for normal result sets, else this response contains a dummy result set

// with no signature nor other data.

RpcMetadata metadata = 7;

}

// Response to PrepareAndExecuteRequest

message ExecuteResponse {

repeated ResultSetResponse results = 1;

bool missing_statement = 2; // Did the request fail because of no-cached statement

RpcMetadata metadata = 3;

}

// Response to PrepareRequest

message PrepareResponse {

StatementHandle statement = 1;

RpcMetadata metadata = 2;

}

// Response to FetchRequest

message FetchResponse {

Frame frame = 1;

bool missing_statement = 2; // Did the request fail because of no-cached statement

bool missing_results = 3; // Did the request fail because of a cached-statement w/o ResultSet

RpcMetadata metadata = 4;

}

// Response to CreateStatementRequest

message CreateStatementResponse {

string connection_id = 1;

uint32 statement_id = 2;

RpcMetadata metadata = 3;

}

// Response to CloseStatementRequest

message CloseStatementResponse {

RpcMetadata metadata = 1;

}

// Response to OpenConnectionRequest {

message OpenConnectionResponse {

RpcMetadata metadata = 1;

}

// Response to CloseConnectionRequest {

message CloseConnectionResponse {

RpcMetadata metadata = 1;

}

// Response to ConnectionSyncRequest

message ConnectionSyncResponse {

ConnectionProperties conn_props = 1;

RpcMetadata metadata = 2;

}

message DatabasePropertyElement {

DatabaseProperty key = 1;

TypedValue value = 2;

RpcMetadata metadata = 3;

}

// Response for Meta#getDatabaseProperties()

message DatabasePropertyResponse {

repeated DatabasePropertyElement props = 1;

RpcMetadata metadata = 2;

}

// Send contextual information about some error over the wire from the server.

message ErrorResponse {

repeated string exceptions = 1; // exception stacktraces, many for linked exceptions.

bool has_exceptions = 7; // are there stacktraces contained?

string error_message = 2; // human readable description

Severity severity = 3;

uint32 error_code = 4; // numeric identifier for error

string sql_state = 5; // five-character standard-defined value

RpcMetadata metadata = 6;

}

message SyncResultsResponse {

bool missing_statement = 1; // Server doesn't have the statement with the ID from the request

bool more_results = 2; // Should the client fetch() to get more results

RpcMetadata metadata = 3;

}

// Generic metadata for the server to return with each response.

message RpcMetadata {

string server_address = 1; // The host:port of the server

}

// Response to a commit request

message CommitResponse {

}

// Response to a rollback request

message RollbackResponse {

}

// Response to a batch update request

message ExecuteBatchResponse {

string connection_id = 1;

uint32 statement_id = 2;

repeated uint64 update_counts = 3;

bool missing_statement = 4; // Did the request fail because of no-cached statement

RpcMetadata metadata = 5;

}

[Adam Cozzette]

I suspect that your build is mixing your downloaded 3.12.3 protobuf source with a different protobuf version installed by your distro's package manager. I would recommend making sure that your project is only compiling against one version of protobuf. Also, I don't think this is related but you might want to use 3.13.0 or 3.12.4 instead of 3.12.3. We made a mistake with 3.12.3 and published tar.gz files on the GitHub release page that were generated from the wrong Git commit.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/98864904-d137-419a-901c-917727bddbe6n%40googlegroups.com.

[Test Last]

Hi 

To try and minimise this sort of thing I installed a brand new virtual box and compiled it on there.

Installing only what I need. The errors changed slightly so I think you are right about that.

However in an attempt to solve the issue I got 3.13.0 and this error was actually generated from that.

Any ideas I can try or more debug info I can give you?

Thanks

[Adam Cozzette]

I still suspect that multiple protobuf versions are being mixed somehow. Are you sure that your virtualbox installation didn't install protobuf?

To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/0233414f-3b95-4d79-8001-cbdc2724e630n%40googlegroups.com.

[Test Last]

Yes I only have one version on there.

If you rebuild a different version and say make install ... surely it should overwrite the previously installed headers and lib right?

Ok I am going to make a fresh installation on Ubuntu and test if protoc is on it. If not I will carry on with my script which I will paste here as well.

I am going to install version 3.13.0

Then give the results.

Thanks

[Adam Cozzette]

Actually a "make install" will not override the headers and library installed by your distro's package manager. The package manager will put those files in /usr/lib and /usr/include, whereas a "make install" will install into /usr/local.

To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/cc700778-8c37-4c1a-9f09-0a345c6abd8fn%40googlegroups.com.

[Test Last]

Ok so this morningI have installed a new ubuntu 20 server with bare minimum.

I checked inside /usr/lib and /usr/include as well as /usr/local/include, /usr/local/lib for any files related but could not see anything even with ls -al | grep proto or google.

Then I copied over my source and the related proto files.

I then ran this build script to prepare the system and install all the libs necessary.

#apt update #This should be used when running in docker containers.Also all SUDOs should be removed

sudo apt install build-essential wget automake autoconf libtool libtool-bin unixodbc unixodbc-dev make unzip g++ git -y

#sudo apt install build-essential automake autoconf2.6.4 libtool libtool-bin unixodbc unixodbc-dev make unzip g++ -y

git clone https://github.com/curl/curl.git curl

cd curl

autoreconf -i

./configure

make -j8

#sudo

sudo make install

cd ..

rm -dr ./curl

wget https://github.com/protocolbuffers/protobuf/releases/download/v3.13.0/protobuf-cpp-3.13.0.tar.gz -O protobuf.tar.gz

tar -xvf ./protobuf.tar.gz

mv ./protobuf-3.13.0 ./protobuf

cd ./protobuf

./configure

make -j8

make check

#sudo

sudo make install

#sudo

sudo ldconfig

cd ../Protobufs

make -f Makefile                <--- This make file will be pasted beneath

cd ../

make -f Makefile

When it gets to the point where I marked in RED it runs this Makefile inside the Protobufs folder.

LDFLAGS = -L/usr/local/lib `pkg-config --cflags --libs protobuf`\

-Wl,--no-as-needed -lgrpc++_reflection -Wl,--as-needed\

-ldl

CXX = g++

#CPPFLAGS += `pkg-config --cflags protobuf`

CXXFLAGS += -std=c++14

DEBUG = -g

GRPC_CPP_PLUGIN = grpc_cpp_plugin

GRPC_CPP_PLUGIN_PATH ?= `which $(GRPC_CPP_PLUGIN)`

all: common request response common.o request.o response.o

common:

protoc -I=. --cpp_out=. ./common.proto

request:

protoc -I=. --cpp_out=. ./request.proto

response:

protoc -I=. --cpp_out=. ./response.proto

common.o: common.pb.cc

$(CXX) $(DEBUG) -fPIC -c $^ $(LDFLAGS) -o $@ $(CXXFLAGS)

request.o: request.pb.cc

$(CXX) $(DEBUG) -fPIC -c $^ $(LDFLAGS) -o $@ $(CXXFLAGS)

response.o: response.pb.cc

$(CXX) $(DEBUG) -fPIC -c $^ $(LDFLAGS) -o $@ $(CXXFLAGS)

clean:

rm -f *.o *.pb.cc *.pb.h client

So after that I have all my protobuf compiled .o files and the lib has already compiled without issues.

The .proto files are paste on the query above but for clarity I will pastebin them here again.

common.proto

https://hastebin.com/dewocipaqe.cpp

response.proto

https://hastebin.com/fagaluxaca.cpp

request.proto

https://hastebin.com/ugutesijuq.cpp

Unfortunately I can not paste you the string I receive that needs parsing cause its bytes, I can however send you a file containing as bytes. Strange thing is this works fine in C++ Windows.

This is my template function I use to parse since every message that comes in must be parsed by WireMessage as per Avatica way of doing things.

template <class Ret>

void parse(std::stringstream* inc, Ret* outClass)

{

WireMessage wm = WireMessage();

wm.ParseFromIstream(inc);

Ret ret = Ret();

outClass->ParseFromString(wm.wrapped_message());

}

Typically I will receive a response and call something like

parse<ExecuteResponse>(&response, PERResponse);

This is a paste of my gdb and a response that usually works on windows. I did delete some of the @s though for data sensitivity.

Program received signal SIGSEGV, Segmentation fault.

0x0000555555641fba in google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<ColumnValue>::TypeHandler> (this=0x55555570fb30, prototype=0x0)

    at /usr/local/include/google/protobuf/repeated_field.h:1744

1744   if (rep_ != NULL && current_size_ < rep_->allocated_size) {

(gdb) backtreace

(gdb) backtrace

#0  0x0000555555641fba in google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<ColumnValue>::TypeHandler> (this=0x55555570fb30, prototype=0x0)

    at /usr/local/include/google/protobuf/repeated_field.h:1744

#1  0x000055555563eee3 in google::protobuf::RepeatedPtrField<ColumnValue>::Add (

    this=0x55555570fb30) at /usr/local/include/google/protobuf/repeated_field.h:2195

#2  0x000055555563c4b4 in Row::_internal_add_value (this=0x55555570fb20) at common.pb.h:5781

#3  0x000055555562dedf in Row::_InternalParse (this=0x55555570fb20, 

    ptr=0x555555717b11 "*\n\023\b\025\032\017Alexandre Rocha\"\023\b\025\032\017Alexandre Rocha\n,\n\024\b\025\032\020aleol.com.br\"\024\b\025\032\020alerol.com.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\226\001\n&\n\021\b\025\032\rAstrid Gruber\"\021\b\025\032\rAber\n8\n\032\b\025\032\026astrid.gru"..., ctx=0x7fffffffd8e0) at common.pb.cc:3927

#4  0x000055555563ff38 in google::protobuf::internal::ParseContext::ParseMessage<Row> (

    this=0x7fffffffd8e0, msg=0x55555570fb20, 

    ptr=0x555555717b10 "\n*\n\023\b\025\032\017Alexandre Rocha\"\023\b\025\032\017Alexandre Rocha\n,\n\024\b\025\032\020aleuol.com.br\"\024\b\025\032\020aleol.com.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\226\001\n&\n\021\b\025\032\rAstrid Gruber\"\021\b\025\032\rAstber\n8\n\032\b\025\032\026astrid.gr"...) at /usr/local/include/google/protobuf/parse_context.h:614

#5  0x000055555562ce6b in Frame::_InternalParse (this=0x555555712b10, 

    ptr=0x555555717b0e "\214\001\n*\n\023\b\025\032\017Alha\"\023\b\025\032\017Alexandre Rocha\n,\n\024\b\025\032\020aleol.com.br\"\024\b\025\032\020alerol.com.br\n\030\n\n\b\025\032\006Brazil\"\n\b\025\032\006Brazil\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\226\001\n&\n\021\b\025\032\rAstrid Gruber\"\021\b\025\032\rAer\n8\n\032\b\025\032\026astrid."..., ctx=0x7fffffffd8e0) at common.pb.cc:3688

#6  0x00005555556872ba in google::protobuf::internal::ParseContext::ParseMessage<Frame> (

    this=0x7fffffffd8e0, msg=0x555555712b10, 

    ptr=0x555555717a72 "\020\001\032\226\001\n(\n\022\b\025\032\016Aaron Mitchell\"\022\b\025\032\016Aaron Mitchell\n8\n\032\b\025\032\026ronoo.ca\"\032\b\025\032\026onhoo.ca\n\030\n\n\b\025\032\006Canada\"\n\b\025\032\006Canada\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\214\001\n*\n\023\b\025\032\017Alexandre Rocha\"\023\b\025\032\017Alexandre Roc"...) at /usr/local/include/google/protobuf/parse_context.h:614

#7  0x000055555566fd16 in ResultSetResponse::_InternalParse (this=0x55555571eeb0, 

    ptr=0x555555717a70 "\266F\020\001\032\226\001\n(\n\022\b\025\032\016Aaron Mitchell\"\022\b\025\032\016Aaron Mitchell\n8\n\032\b\025\032\02rhoo.ca\"\032\b\025\032\026aoo.ca\n\030\n\n\b\025\032\006Canada\"\n\b\025\032\006Canada\n\026\n\t\b\032\032\005\063\067.62\"\t\b\032\032\005\063\067.62\032\214\001\n*\n\023\b\025\032\017Alexandre Rocha\"\023\b\025\032\017Alexandre R"..., ctx=0x7fffffffd8e0) at response.pb.cc:788

#8  0x00005555556875b6 in google::protobuf::internal::ParseContext::ParseMessage<ResultSetResponse>

    (this=0x7fffffffd8e0, msg=0x55555571eeb0, 

    ptr=0x5555557178e3 "\n\bAK647078\020j\030\001\"\373\002\n\\ \001\060\001@\377\001J\bFullnameR\bFullnameZ\adefault`\377\001r\aVidDemoz\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\006STRING\030\025\nX\b\001 \001\060\001@\377\001J\005EmailR\005EmailZ\adefault`\377\001r\aVidDemoz\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\006STRING\030"...)

    at /usr/local/include/google/protobuf/parse_context.h:614

#9  0x0000555555671578 in ExecuteResponse::_InternalParse (this=0x7fffffffdd60, 

    ptr=0x5555557178e1 "\366I\n\bAK647078\020j\030\001\"\373\002\n\\ \001\060\001@\377\001J\bFullnameR\bFullnameZ\adefault`\377\001r\aVidDemoz\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\006STRING\030\025\nX\b\001 \001\060\001@\377\001J\005EmailR\005EmailZ\adefault`\377\001r\aVidDemoz\005Spark\200\001\001\232\001\020java.lang.String\242\001\f\b\f\022\006STRIN"..., 

    ctx=0x7fffffffd8e0) at response.pb.cc:1135

#10 0x00007ffff7d9be14 in google::protobuf::internal::MergeFromImpl<false> (input=..., 

    msg=0x7fffffffdd60, parse_flags=google::protobuf::MessageLite::kParse)

    at ./google/protobuf/parse_context.h:224

#11 0x000055555557e283 in proto::helper::parseNonStream<ExecuteResponse> (inc=0x7fffffffdb50, 

    outClass=0x7fffffffdd60) at transProto.h:34

#12 0x0000555555579292 in ConnectEngine::PrepareAndExecuteReq (this=0x7fffffffe190, 

    query="SELECT * from VidDemo", PERResponse=0x7fffffffdd60, maxRows=9223372036854775807, 

    frameMaxSize=2147483647, _statementId=106) at ConnectEngine.cpp:662

#13 0x00005555555858e8 in Consolidation::executeQuery (this=0x7fffffffe0d0, 

    query="SELECT * from VidDemo") at Consolidation.cpp:173

#14 0x000055555556c3b2 in main () at testapp.cpp:44

(gdb) 

!!NOTE ============================================================================================================

Something I have just noticed is that the error seems to change when I implement that which is done inside the template function normally.

Which is very odd to me but the error I get when doing so is.

[libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field 'PrepareAndExecuteRequest.connection_id' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.  
[libprotobuf FATAL /usr/local/include/google/protobuf/repeated_field.h:1730] CHECK failed: (index) < (current_size_):  
terminate called after throwing an instance of 'google::protobuf::FatalException'
 what():  CHECK failed: (index) < (current_size_):  
Aborted (core dumped)

That error is clearly not the case since I can do the exact same test in my windows implementation and it works fine.

[Test Last]

Hi

Ok so I have found the problem.

The problem was that I had a class name for one of my classes that was already defined in one of the libs.

Therefore inside the header it was trying to use the wrong class inside the repeated_field parser.

For obv reasons then got the Seg Fault!