2GB protobuf limit

[Gareth Williams]

Hi there,

I'm trying to compile a state space model and I believe I'm hitting the protobuf message limit of 2GB when using `@tf.function`. I printed the message length just before the error and it's 2217676732, so it makes sense that it's hitting the 2GB limit.

I'm just wondering, is there any way to get around this that anyone knows of? I read about Cap'n Proto here https://stackoverflow.com/a/34186672 but presumably I can't just connect this to TensorFlow. 

Is there any way to compile a model and somehow use smaller messages? Any suggestions appreciated.

Many thanks,

Gareth

(error is

  File "...python3.9/site-packages/tensorflow/python/eager/function.py", line 497, in __init__
    function_def.ParseFromString(compat.as_bytes(proto_data))
google.protobuf.message.DecodeError: Error parsing message with type 'tensorflow.FunctionDef'

)