Issue 469 in protobuf: how to write/read more than one piece of message to/from a text file in c++

[prot...@googlecode.com]

Status: New
Owner: liu...@google.com
Labels: Type-Defect Priority-Medium

New issue 469 by wuy...@gmail.com: how to write/read more than one piece of
message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

What steps will reproduce the problem?
1.fullfill a message
2.invoke SerializeToString method of this message;
3.open a file with combined mode of "std::ios::app|std::ios::binary" and
write the data to this file and close file
4.open file stored the messages in "std::ios::in|std::ios::binary", read a
line,and invoke ParseFromString, this ParseFromString will fail.

What is the expected output? What do you see instead?
expected: a parsed message
fact:libprotobuf ERROR ..\src\google\protobuf\message_lite.cc:123] Can't
parse message of type "FileState" because it is missing required
fields:name,state

What version of the product are you using? On what operating system?
version:protobuf-2.4.0a
system: windows7 x86

Please provide any additional information below.
MESSAGE definition:
enum FileAction {
FILE_ACTION_ADD = 0;
FILE_ACTION_DEL = 1;
FILE_ACTION_MODIFY = 2;
FILE_ACTION_RENAME = 3;
}

message FileState {
required string name = 1; // file name
required FileAction state = 2; // file state
}
the test source file will be found in the attached file


Attachments:
mail.cpp 1.1 KB

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[prot...@googlecode.com]

Comment #1 on issue 469 by wuy...@gmail.com: how to write/read more than

one piece of message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

(No comment was entered for this change.)

Attachments:
mail.cpp 1.2 KB

[prot...@googlecode.com]

Comment #2 on issue 469 by wuy...@gmail.com: how to write/read more than

one piece of message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

Noting
this,https://developers.google.com/protocol-buffers/docs/cpptutorial?hl=zh-CN
gives a example of read/write multiple message from/to a file, but a better
way is to allow user to read/write message from/to file at any time.

[prot...@googlecode.com]

Updates:
Status: Invalid
Owner: xiaof...@google.com

Comment #3 on issue 469 by xiaof...@google.com: how to write/read more than

one piece of message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

You must delimiter your messages by some other means rather than "lines".
Reading "a line" from a binary file is almost always wrong.
Serialized messages are just binary data chunks. You can use whatever means
you use to read/write multiple data chunks in a file to read/write multiple
messages.

[prot...@googlecode.com]

Comment #4 on issue 469 by wuy...@gmail.com: how to write/read more than

one piece of message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

Yes,it is just binary data chunk. When I open the file stored some protobuf
messages, it seems like that the SerializeToString method forms a c++
string with a "\n" as the last character. And when debugging, the first
character of the serielized string is '\n' whose ASCII code is 0x0A.
if you try parse it in this way,it will work: head a character '\n' at the
buffer that receive the parse binary data chunk gotten from readline.
At most of the time, it will work, yet there is a problem: when it failed
to parse, you can do nothing, readint its source code, it's a lot of work.
And I have come across this problem. In other words, the parsing method by
heading a character '\n' is unreliable or unstable.

Any one have other Feasible way to read/write messages from/to a file?
Thanks.

[prot...@googlecode.com]

Comment #5 on issue 469 by wuy...@gmail.com: how to write/read more than

one piece of message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

Yes,it is just binary data chunk. When I open the file stored some protobuf
messages, it seems like that the SerializeToString method forms a c++

string with '\n' as the last character. And when debugging, the first

character of the serielized string is '\n' whose ASCII code is 0x0A.
if you try parse it in this way,it will work: head a character '\n' at the

buffer that receive the parsed binary data chunk gotten from readline.the
c++ example code is like:
string fPath("message.txt");
string strMsg;
char buf[1024] = {0};
fstream f;

f.open(fPath.c_str(),std::ios_base::in | std::ios_base::binary);
f.seekg(std::ios_base::beg);
while(!f.eof())
{
buf[0] = '\n';
f.getline(buf+1,sizeof(buf) - sizeof(char));
string strTmp(buf);
if(!strTmp.empty())
{
msg.ParseFromString(buf);
//msg.ParsePartialFromString(strTmp);
cout<<"name:\t\t"<<msg.name()<<std::endl;
cout<<"state:\t\t"<<static_cast<int>(msg.state())<<std::endl;
}
msg.Clear();
memset(buf,'\0',sizeof(buf));

}

At most of the time, it will work, yet there is a problem: when it failed

to parse, you can do nothing, reading source code, it's a much lot of work.

[prot...@googlecode.com]

Comment #6 on issue 469 by wuy...@gmail.com: how to write/read more than

one piece of message to/from a text file in c++
http://code.google.com/p/protobuf/issues/detail?id=469

As said by https://developers.google.com/protocol-buffers/docs/techniques,
the following code is passed to test.
Generally,it's a way that you yourself encode the string and decode it.
#include "GFileState.pb.h"
#include <fstream>
#include <string>
#include <iostream>
using std::string;
using std::fstream;
using std::cout;
int main(int argc,char* argv[])
{

string fPath("message.txt");
string strMsg;
char buf[1024] = {0};
fstream f;

int size;
f.open(fPath.c_str(),std::ios_base::app | std::ios_base::binary);
FileState msg,msg2;
msg.set_name("D:\\Test1");
msg.set_state(FILE_ACTION_ADD);
msg.SerializeToString(&strMsg);
//msg.SerializePartialToString(&strMsg);
size = strMsg.length();
f.write((char*)&size,sizeof(size));
f.write(strMsg.c_str(),size);
f.seekg(std::ios_base::end);

msg.set_name("/usr/home/nc/download");
msg.set_state(FILE_ACTION_MODIFY);
msg.SerializeToString(&strMsg);

size = strMsg.length();
f.write((char*)&size,sizeof(size));
f.write(strMsg.c_str(),size);
f.close();

f.open(fPath.c_str(),std::ios_base::in | std::ios_base::binary);
f.seekg(std::ios_base::beg);

strMsg.clear();
size = 0;
while(!f.eof())
{
f.read((char*)&size,sizeof(size));
if(size > 0 && size < sizeof(buf))
{
f.read(buf,size);
msg.ParseFromString(buf);

cout<<"name:\t\t"<<msg.name()<<std::endl;
cout<<"state:\t\t"<<static_cast<int>(msg.state())<<std::endl;
}
msg.Clear();
memset(buf,'\0',sizeof(buf));

size = 0;
}
f.close();
std::cin >>strMsg;
return 0;