Proposal: Adding extra fields to RequestMetadata

[Daniel Wagner-Hall]

Currently, remote_execution.proto specifies a RequestMetadata protobuf, optionally used as a header to provide the remote execution server with metadata about a build.

I’d like to add a few optional fields to it:

• string target_label (e.g. //your:target)
• string action_mnemonic (e.g. Javac)
• string workspace_name (whatever the value of the name attribute is in the call to workspace() in the WORKSPACE file)

And have Bazel set them as appropriate. Mostly this is for debug metadata, but I imagine they could also be used for things like naive cost estimation, or even preferential worker routing.

Any thoughts?

[Eric Burnett]

+1! More metadata seems generally useful, and forge has similar in their logs which has proven quite useful in practice.

On the off chance you're not already planning for it, I'll request we either come up with generic descriptions of these fields so they clearly map onto non-bazel tools, or generic renames of the same. I don't want to have to lean on bazel knowledge for how to interpret the metadata in the REAPI.

• 'action_mnemonic' seems fine and easy to describe.
• 'target_label': If we can succinctly explain what a "target" is, might be fine, otherwise maybe "action label"? Though this calls into question how to map it in when bazel sends multiple actions for a single target, e.g. compile and link steps. Should those be labeled the same, distinct ("//foo/bar compile"), or be split into multiple fields like "label" and "sublabel"? At any rate, we should be clear if the expectation is that a target_label maps to at most one action execution (and so is 1:1 with action_id, and probably better named action_label), or may encompass multiple.
•  'workspace_name': similarly, fine if we can come up with a generic description of what a "workspace" is in this context, otherwise possibly might need to be genericized.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Remote Execution APIs Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to remote-execution...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/remote-execution-apis/0291D6B1-F729-4D66-85FA-1C35EE4910D9%40apple.com.

[George Gensure]

Further +1. This is conceptually one of our regalia: association between targets as action groupings and actions.

On Fri, Feb 5, 2021 at 10:36 AM 'Eric Burnett' via Remote Execution APIs Working Group <remote-exe...@googlegroups.com> wrote:

• 'target_label': If we can succinctly explain what a "target" is, might be fine, otherwise maybe "action label"? Though this calls into question how to map it in when bazel sends multiple actions for a single target, e.g. compile and link steps. Should those be labeled the same, distinct ("//foo/bar compile"), or be split into multiple fields like "label" and "sublabel"? At any rate, we should be clear if the expectation is that a target_label maps to at most one action execution (and so is 1:1 with action_id, and probably better named action_label), or may encompass multiple.

My vote is for correlated_actions_id if we can't stomach the bazelization of the REAPI, consistent with our correlated_invocations_id.

•  'workspace_name': similarly, fine if we can come up with a generic description of what a "workspace" is in this context, otherwise possibly might need to be genericized.

'invocation_namespace_id'? It essentially creates a boundary between what is in and out of an amorphous set of resources, not uniquely declared across clients.

-George

Thanks!

On Fri, Feb 5, 2021 at 4:47 AM 'Daniel Wagner-Hall' via Remote Execution APIs Working Group <remote-exe...@googlegroups.com> wrote:

Currently, remote_execution.proto specifies a RequestMetadata protobuf, optionally used as a header to provide the remote execution server with metadata about a build.

I’d like to add a few optional fields to it:

• string target_label (e.g. //your:target)
• string action_mnemonic (e.g. Javac)
• string workspace_name (whatever the value of the name attribute is in the call to workspace() in the WORKSPACE file)

And have Bazel set them as appropriate. Mostly this is for debug metadata, but I imagine they could also be used for things like naive cost estimation, or even preferential worker routing.

Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Remote Execution APIs Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to remote-execution...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/remote-execution-apis/0291D6B1-F729-4D66-85FA-1C35EE4910D9%40apple.com.

--
You received this message because you are subscribed to the Google Groups "Remote Execution APIs Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to remote-execution...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/remote-execution-apis/CAJZTgz2QSBFG19SOOLEKrLzB9WjpPfrC_svwtsk6SJObwTFhyg%40mail.gmail.com.

[Daniel Wagner-Hall]

On 5 Feb 2021, at 15:36, Eric Burnett <ericb...@google.com> wrote:

+1! More metadata seems generally useful, and forge has similar in their logs which has proven quite useful in practice.

On the off chance you're not already planning for it, I'll request we either come up with generic descriptions of these fields so they clearly map onto non-bazel tools, or generic renames of the same. I don't want to have to lean on bazel knowledge for how to interpret the metadata in the REAPI.

• 'action_mnemonic' seems fine and easy to describe.
• 'target_label': If we can succinctly explain what a "target" is, might be fine, otherwise maybe "action label"? Though this calls into question how to map it in when bazel sends multiple actions for a single target, e.g. compile and link steps. Should those be labeled the same, distinct ("//foo/bar compile"), or be split into multiple fields like "label" and "sublabel"? At any rate, we should be clear if the expectation is that a target_label maps to at most one action execution (and so is 1:1 with action_id, and probably better named action_label), or may encompass multiple.

In Bazel, there are two axes of divergence within a target: Compile vs Link (which is the mnemonic), and sub-target files (e.g. cc_library may have a CppCompile per file, and a single CppLink). I believe the two primary ways Bazel differentiates these is:

1. The "Primary" output file of the action (which is already known from the contents of the Action, so doesn't need extra metadata).

2. The "Progress message" of the action (which is optional, not reliably set, is freeform, has no guarantees around format or uniqueness, and can be very verbose).

I think "target" referring to the high-level concept that you'd see in a BUILD file or invoke on the command line is standalone useful (and AFAIK most build systems refer to this as a "target"), but I don't think that's sufficiently unique for the at-most-one you're describing... Is there something you're particularly hoping to achieve wit the at-most-one relationship?

Dropping "label" seems good though, that's very much a bazelification. But I think the crux here is that it's something bigger than an action, so either stepping back ("target"), or as George suggested, showing it's a way of linking actions ("correlated_actions_id"), feel like good directions.

[Daniel Wagner-Hall]

Let the bike-shedding commence: https://github.com/bazelbuild/remote-apis/pull/186