Re: error when reading the config file(protobuf)

[Deanna Garcia]

I think this is more likely a TensorFlow question so I would ask there.

On Saturday, January 22, 2022 at 4:30:39 AM UTC-8 caixia...@gmail.com wrote:

Hello, everyone, I'm currently trying to retrain the panoptic_deeplab using custom dataset. However, I got stuck due to the error shown in the following, could you please help me to figure out what the problem could be? It seems that something goes wrong in the config file.

python trainer/train.py \ --config_file=/home/caixiaoni/Desktop/project/deeplab2/configs/metal_part/panoptic_deeplab/resnet50_os32_semseg.textproto \ --mode=eval \ --model_dir=/home/caixiaoni/Desktop/project/metal_part_retrain_1 \ --num_gpus=0 2022-01-18 23:36:30.240123: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 I0118 23:36:31.325175 139693994784576 train.py:65] Reading the config file. Traceback (most recent call last): File "trainer/train.py", line 76, in <module> app.run(main) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "trainer/train.py", line 67, in main config = text_format.ParseLines(proto_file, config_pb2.ExperimentOptions()) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 759, in ParseLines return parser.ParseLines(lines, message) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 812, in ParseLines self._ParseOrMerge(lines, message) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 835, in _ParseOrMerge tokenizer = Tokenizer(str_lines) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 1255, in __init__ self._SkipWhitespace() File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 1283, in _SkipWhitespace self._PopLine() File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 1272, in _PopLine self._current_line = next(self._lines) File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/google/protobuf/text_format.py", line 832, in <genexpr> str_lines = ( File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 206, in __next__ retval = self.readline() File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 170, in readline self._preread_check() File "/home/caixiaoni/anaconda3/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 79, in _preread_check self._read_buf = _pywrap_file_io.BufferedInputStream( TypeError: __init__(): incompatible constructor arguments. The following argument types are supported: 1. tensorflow.python.lib.io._pywrap_file_io.BufferedInputStream(filename: str, buffer_size: int, token: tensorflow.python.lib.io._pywrap_file_io.TransactionToken = None) Invoked with: None, 524288

The config file looks like:

experiment_name: "coco_test"

model_options {

# Update the path to the initial checkpoint (e.g., ImageNet

# pretrained checkpoint).

initial_checkpoint: "/home/caixiaoni/Desktop/project/resnet50_os16_panoptic_deeplab_coco_train/ckpt-200000"

backbone {

name: "resnet50"

output_stride: 16

}

decoder {

feature_key: "res5"

decoder_channels: 256

aspp_channels: 256

atrous_rates: 6

atrous_rates: 12

atrous_rates: 18

}

panoptic_deeplab {

low_level {

feature_key: "res3"

channels_project: 64

}

low_level {

feature_key: "res2"

channels_project: 32

}

instance {

low_level_override {

feature_key: "res3"

channels_project: 32

}

low_level_override {

feature_key: "res2"

channels_project: 16

}

instance_decoder_override {

feature_key: "res5"

decoder_channels: 128

atrous_rates: 6

atrous_rates: 12

atrous_rates: 18

}

center_head {

output_channels: 1

head_channels: 32

}

regression_head {

output_channels: 2

head_channels: 32

}

}

semantic_head {

output_channels: 134

head_channels: 256

}

}

}

trainer_options {

save_checkpoints_steps: 1000

save_summaries_steps: 100

steps_per_loop: 100

loss_options {

semantic_loss {

name: "softmax_cross_entropy"

weight: 1.0

top_k_percent: 0.2

}

center_loss {

name: "mse"

weight: 200

}

regression_loss {

name: "l1"

weight: 0.01

}

}

solver_options {

base_learning_rate: 0.0005

training_number_of_steps: 200000

warmup_steps: 2000

}

}

train_dataset_options {

dataset: "metal_part"

# Update the path to training set.

file_pattern: "/home/caixiaoni/Desktop/project/part-TFRecord/train-*.tfrecord"

# Adjust the batch_size accordingly to better fit your GPU/TPU memory.

# Also see Q1 in g3doc/faq.md.

batch_size: 16

crop_size: 513

crop_size: 513

min_resize_value: 513

max_resize_value: 513

augmentations {

min_scale_factor: 0.5

max_scale_factor: 1.5

scale_factor_step_size: 0.1

autoaugment_policy_name: "simple_classification_policy_magnitude_scale_0.2"

}

increase_small_instance_weights: true

small_instance_weight: 3.0

}

eval_dataset_options {

dataset: "metal_part"

# Update the path to validation set.

file_pattern: "/home/caixiaoni/Desktop/project/part-TFRecord/val-*.tfrecord"

batch_size: 1

crop_size: 513

crop_size: 513

min_resize_value: 513

max_resize_value: 513

# Add options to make the evaluation loss comparable to the training loss.

increase_small_instance_weights: true

small_instance_weight: 3.0

}

evaluator_options {

continuous_eval_timeout: -1

stuff_area_limit: 4096

center_score_threshold: 0.1

nms_kernel: 41

save_predictions: true

save_raw_predictions: false

# Use pure tf functions (i.e., no CUDA kernel) to merge semantic and

# instance maps. For faster speed, compile TensorFlow with provided kernel

# implementation under the folder `tensorflow_ops`, and set

# merge_semantic_and_instance_with_tf_op to true.

merge_semantic_and_instance_with_tf_op: false

}

I currently used ubuntu-18.04 and have tried with the same code but different path using MacOS system, there isn't error when reading the config file but Ubuntu has. But Mac doesn't have Nvidia GPU such that I couldn't use.

I assume the problem in the config file, could anyone give me some suggestions? I appreciate for any hints! Thanks in advance!