Condense port_def.inc

[Evgenii Kliuchnikov]

Hello.

 Just today I've faced the problem: malformed or corrupted precompiled file: 'ran out of source locations'

The biggest entity (~15% of address space): third_party/protobuf/port_def.inc:928:25: note: file entered 8835 times using 339917790B (339.92MB) of space plus 4744395B (4.74MB) for macro expansions

What I propose: develop port_def.inc in port_def.full.inc but distill it for compilation purposes (drop whitespace and comments; ideally bake fixed defines).

WDYT? If there are any supporters (i.e. chances to get this change approved and landed) I can start writing CL / PR right away.

Best regards,

  Eugene.

[Em Rauch]

I believe the root cause of your issue is most likely that you have deep .proto import chains and so your .pb.h files are transitively so large that its a problem, and I think reducing the whitespace in port_def would not buy you a meaningful amount of runway on that property (if you're _exactly_ hitting a limit it'll look it unbreaks you, but then you add a few more .proto files organically and you'll be back where you started).

If that's the root cause, I think realistically may just need to improve your hygiene to reduce how transtively deep your schema import chains are to mitigate that problem.

Internally inside Google we have support for a different kind of header that we emit as .proto.h instead of .pb.h which are the symbols for one .proto file and only forward declarations of the next level down, which mitigates this problem for deep chains, but we haven't gotten around to open sourcing it (the ergonomics of it are worse than .pb.h behavior, and such deep import chains tend to be rare outside of our monorepo so there's not as much demand for it).  If that would be valuable to you, please feel free to open a feature request on our github and we can try to get it somewhere on our schedule to offer a better affordance to avoid such degenerate import chains hitting clang limits.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/protobuf/ce27670f-90a6-4bb7-8aef-ff96635c7e74n%40googlegroups.com.