Understanding of BES data transmissions

[Michael Fitzner]

We would like to better understand what kind of data are transferred via BES upload e.g. blobs and when this is typically happening and how long does it take.

For that reason I would have the following questions:

- Is there a way to get further details about the BES transmissions e.g. Bazel profile

- Is there a way to log the traffic of the BES transmissions e.g. amount of bytes sent

Thanks in advance!

[Brian Saghy]

Hi Michael,

The BEP Examples documentation and BEP Glossary are helpful to give an overview of BEP content. If you'd like to see the actual information that is sent for a given build to your BES backend, you can use the --build_event_text_file flag with an output file param with your bazel build/test commands. This will generate a text file of the raw BEP contents. It's not the most human readable, but is very useful for seeing what data was uploaded. Note that the Bazel profile and other artifacts like test.xml/logs are not uploaded to the BES backend directly, but to the remote cache (if configured via --remote_cache). The BEP may reference those artifacts.

I'm not aware of any built-in Bazel flags that will show you the BES transmission traffic on the client side. You could perhaps put a proxy in between your bazel client and BES backend to monitor the traffic and upload times, or use a network traffic monitoring tool on the host. 

Hope this helps!

Brian Saghy
Product Owner
Gradle
W. gradle.com
Gradle

[Michael Fitzner]

Hi Brian,

Thanks for your feedback.

https://bazel.build/advanced/performance/build-performance-metrics and https://bazel.build/advanced/performance/build-performance-breakdown describe some technics to gather build metrics (also network) but regarding BEP/BES I couldn't see any specific metrics.

Perhaps a proxy could help to get more insights.

Best regards,
Michael