Fail to parse the binary file to json format file

[Chunting Jiao]

Hi there,

It seems strange to me that I couldn't generate json format file from flatbuffer binary file correctly.

Nothing is complicated in this part and with the same code, I did it in another project.

I'm here to seek your opinions about what the possible reasons are?

The verifier returns true.

flatbuffers::Verifier verifier(buf, bufsize);

grl::flatbuffer::VerifyKUKAiiwaStatesBuffer(verifier)

fbb.GetSize() seems also normal, 3147672 bytes.

The schema file is also loaded correctly by 

flatbuffers::LoadFile(fbs_filename.c_str(), false, &schemafile);

(I printed out the string schemafile, the result was correct.)

But in this line,

OK = OK && flatbuffers::GenerateText(parser, buf, &jsongen);

the return value is true, but string jsogen only contains "{/n}/n", the result here is totally different from expectations.

I also run the command to generate json file,

flatc -I . --json  KUKAiiwa.fbs -- Kuka_test_binary.iiwa

It didn't work either.

Attached you can find my fbs file, binary file and the generated json file.

Thanks.
Chunting

   

flatbuffers::Offset<grl::flatbuffer::KUKAiiwaStates> kukaStates = grl::toFlatBuffer(fbb, kukaiiwaStateVec);

grl::flatbuffer::FinishKUKAiiwaStatesBuffer(fbb, kukaStates);

uint8_t *buf = fbb.GetBufferPointer();

std::size_t bufsize = fbb.GetSize();

for (flatbuffers::uoffset_t i = 0; i < bufsize; i++) {

printf("%d ", buf[i]);

}

flatbuffers::Verifier verifier(buf, bufsize);

std::string binary_file_path = "Kuka_test_binary.iiwa";

std::string json_file_path = "kuka_test_text.json";

OK = flatbuffers::SaveFile(binary_file_path.c_str(), reinterpret_cast<const char *>(buf), bufsize, true);

// Get the current working directory

std::string fbs_filename("KUKAiiwa.fbs");

std::cout << "fbs exist? " << flatbuffers::FileExists(fbs_filename.c_str()) << std::endl;

flatbuffers::Parser parser;

std::vector<std::string> includePaths = std::vector<std::string >();

std::string current_working_dir;

char buff[2048];

/// Get the current working directory

getcwd(buff, 2048);

current_working_dir = std::string(buff);

std::cout << "The current working dir: " << current_working_dir << std::endl;

includePaths.push_back(current_working_dir);

std::string schemafile;

flatbuffers::LoadFile(fbs_filename.c_str(), false, &schemafile);

std::cout<<"Schema File: " << std::endl << schemafile << std::endl;

// parse fbs schema, so we can use it to parse the data after

// create a list of char* pointers so we can call Parse

std::vector<const char *> include_directories;

for(int i = 0; i < includePaths.size(); i++){

include_directories.push_back(includePaths[i].c_str());

}

include_directories.push_back(nullptr);

OK = OK && parser.Parse(schemafile.c_str(), &include_directories[0]);

std::string jsongen;

// now generate text from the flatbuffer binary

OK = OK && flatbuffers::GenerateText(parser, buf, &jsongen);

std::cout << "json string :" << jsongen.size() << std::endl;

std::cout << jsongen.c_str() << std::endl;

// Write the data get from flatbuffer binary to json file on disk.

std::cout << "buffer :" << charP_size(reinterpret_cast<const char *>(buf)) << std::endl;

std::ofstream out(json_file_path);

out << jsongen.c_str();

out.close();

std::cout << "Save json file correctly? " << OK << " Buffer size saved to binary file: " << bufsize << std::endl;

std::cout << "End of the program" << std::endl;

 
 

[Wouter van Oortmerssen]

A file can be corrupt and the verifier can still succeed, for example when a file starts with a bunch of zeroes, it is equivalent to an empty root table to the verifier.

In this case it is not empty though:

-$ hexdump -C ~/Downloads/Kuka_test_binary.iiwa |head

00000000  0c 00 00 00 69 69 77 61  00 00 00 00 dc f4 cf ff  |....iiwa........|

00000010  66 04 d0 ff 00 00 01 01  24 00 00 00 30 00 00 00  |f.......$...0...|

00000020  54 00 00 00 7c 0e 00 00  4c 02 00 00 70 00 00 00  |T...|...L...p...|

00000030  00 6c ec 52 9d 34 a9 42  00 00 00 00 09 00 00 00  |.l.R.4.B........|

00000040  52 6f 62 6f 74 69 69 77  61 00 00 00 21 00 00 00  |Robotiiwa...!...|

00000050  77 68 65 72 65 20 74 68  69 73 20 6d 65 73 73 61  |where this messa|

00000060  67 65 20 69 73 20 67 6f  69 6e 67 20 28 55 52 49  |ge is going (URI|

00000070  29 00 00 00 22 00 00 00  77 68 65 72 65 20 74 68  |)..."...where th|

00000080  69 73 20 6d 65 73 73 61  67 65 20 63 61 6d 65 20  |is message came |

00000090  66 72 6f 6d 20 28 55 52  49 29 00 00 6e 13 d0 ff  |from (URI)..n...|

So we have a root table offset at 12 bytes, a correct file_identifier. The root table is supposed to start with a vtable offset which here is 0xFFCFF4DC, which indicates the vtable is stored very far into the file (which is possible, since this is a big file). But it also means it is shared with another table which typically isn't the case for root tables, since they're usually unique. Unless it is indeed an empty root table.

The above code does not include how you construct the buffer. You should check your call to CreateKUKAiiwaStates() which somehow is creating an empty table.

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.