Hi, I am not a direct user of protobuf, I was training my model with tensorflow using Python, while attempting to save my checkpoint model, this error related to protobuf occurred:
[libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field '
tensorflow.TensorShapeProto.Dim.name' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.
Traceback (most recent call last):
File "train.py", line 77, in <module>
train(model)
File "train.py", line 24, in train
model.optimize()
File "/user-data/HyperBox-main/script/model/box_model.py", line 329, in optimize
self.save_model(itr)
File "/user-data/HyperBox-main/script/model/box_model.py", line 140, in save_model
self.saver.save(self.sess, filename)
File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py", line 1208, in save
self.export_meta_graph(
File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py", line 1254, in export_meta_graph
graph_def=ops.get_default_graph().as_graph_def(add_shapes=True),
File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3345, in as_graph_def
result, _ = self._as_graph_def(from_version, add_shapes)
File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3262, in _as_graph_def
graph.ParseFromString(compat.as_bytes(data))
google.protobuf.message.DecodeError: Error parsing message
from the information above, we can see that:
1) some so-called 'invalid UTF-8 data' was sent to protobuf;
2) Use the 'bytes' type if you intend to send raw bytes.
the function at the second to last line compat.as_bytes(data) is defined as:
def as_bytes(bytes_or_text, encoding='utf-8'):
if isinstance(bytes_or_text, bytearray):
return bytes(bytes_or_text)
elif isinstance(bytes_or_text, _six.text_type): ##_six.text_type = unicode
return bytes_or_text.encode(encoding)
elif isinstance(bytes_or_text, bytes):
return bytes_or_text
else:
raise TypeError('Expected binary or unicode string, got %r' %
(bytes_or_text,))
we can see that this function makes sure that what will be passed into graph.ParseFromString(which later passed to protobuf) is definitely utf-8 data or bytes, however, the protobuf keeps claiming the data passed to it is not valid utf-8 data. Taking one step back, even if the data was not valid utf-8, it is definitely bytes, which it should take in normally.
And I printed some information before the execution of graph.ParseFromString(compat.as_bytes(data)):
pprint(type(data))
graph.ParseFromString(compat.as_bytes(data))
and here's the result:
<class 'bytes'>
the data has already been bytes even before it is passed into compat.as_bytes!
I am sorry for sounding emotional, it's just that I've been stuck with this problem for too long and it really gets me frustrated.