I think there are a couple different ways to avoid calling ByteSize() more than once:
1. You can call ByteSize() yourself and save the result, and then call SerializeWithCachedSizesToArray(). The serialization will use the cached size and thus will not recompute it.
2. You can call SerializeToArray() and then (assuming it was successful) call GetCachedSize() afterward to find out how many bytes were written.
Would one of those help for your use case? It seems to me like you would probably want to go with option 1 since in practice you probably need to know how large the message is so that you can ensure your buffer is large enough before you attempt the serialization.