2GB protobuf limit

Skip to first unread message

Gareth Williams

Apr 9, 2022, 10:04:13 AM4/9/22
to dis...@tensorflow.org
Hi there,

I'm trying to compile a state space model and I believe I'm hitting the protobuf message limit of 2GB when using `@tf.function`. I printed the message length just before the error and it's 2217676732, so it makes sense that it's hitting the 2GB limit.

I'm just wondering, is there any way to get around this that anyone knows of? I read about Cap'n Proto here https://stackoverflow.com/a/34186672 but presumably I can't just connect this to TensorFlow. 

Is there any way to compile a model and somehow use smaller messages? Any suggestions appreciated.

Many thanks,

(error is

  File "...python3.9/site-packages/tensorflow/python/eager/function.py", line 497, in __init__
google.protobuf.message.DecodeError: Error parsing message with type 'tensorflow.FunctionDef'

Reply all
Reply to author
0 new messages