(James Duncan pointed me here and asked to share a bit; note I deal with Python, not Java, though)
If I understand correctly, the basic thing you're trying to do is something like a plugin model, right? Your rule provides the base framework ("invoker" attribute), users provide the plugin ("tool" attribute), and then when your tool runs, it loads the plugin code as part of its execution?
Here's a few ideas on how to make this work: define a new binary target with the additional dependencies, or use whatever Java-specific mechanism there is to tell a binary "here's extra stuff", or (maybe, not 100% sure) manually merge and creating your own runfiles tree.
1. Defining a new binary with extra deps. This is how you get the library into the binary's runfiles. This is the most sound solution. The drawback is defining a new binary for every usage doesn't scale as well (each usage is a unique binary, even if some are conceptually identical, so the aggregate cost of building adds up when it gets wide usage). To do this, during the macro phase, just define a java_binary and concat the extra dep on:
def invoke(name, tool, ...):
java_binary(name=name + "_invoker", deps = [...] + [tool])
_invoker(..., invoker=name + "_invoker")
This is the most sound because it avoids dynamically loading code. Dynamically loading code isn't wrong/bad, it just tends to have more edge cases that might fail or require more special care. Binaries are happiest when they know all the dependencies they need at build time.
This is also the only way to get "tool" directly into invoker's runfiles.
This is also the easiest since your get a Target in your rule, and can then just pass files_to_run along, and don't have to deal with any of the runfiles materializing the rest of this email talks about.
1b. Use a custom rule to merge the runfiles and symlink the executable.
I just thought of this one; I'm not entirely sure if this'll work, but you can try writing a rule to merge the runfiles and symlink the executable. Something like this:
invoker = ctx.attr._invoker
runfiles = invoker[DI].default_runfiles.merge(ctx.attr.tool[DI].default_runfiles)
return [DefaultInfo(executable=executable, runfiles=runfiles)]
_merged_binary = rule(executable = True, ...)
def invoke(..., tool):
_merged_binary(name=name + "_invoker", tool=tool)
_invoker(invoker=name + "_invoker")
2. Saying "here's extra stuff" to a binary. How and if this can work is particular to how the Java binary and library rules work. But the basic idea is to tell the binary the location of the extra stuff using e.g. a flag or environment variable. If that extra stuff is just a file (e.g. a .jar), this should be pretty simple, e.g. just pass a flag: `args.add("--plugin", ctx.file.tool)`, or equiv. Were this Python, the gist would be something that ultimately modifies sys.path; idk what the Java equivalent is.
If it has to be a directory of all the runfiles as would be normally seen, then see (3)
3. Manually merging and materializing runfiles
You can basically mimick what Bazel does under the hood had (1) been done. So e.g. write starlark code to traverse the runfiles, declare_file and symlink for each file, put it all in a depset, calculate the logical runfiles root of your files, then pass that along to the action and tool. Using a fileset might be possible, too; not sure. Obviously, this means you have to flatten the depsets of the runfiles, so those usual caveats apply. I had to this once, and it's fairly straight forward, but kind of tedious/a pain; I was just packaging things into a zip file, though, I never tried to run anything out of it. You might have to fiddle more to make that work, idk.
Issue 15486 is sort of related to this. When I filed that, I wasn't thinking of merging arbitrary runfiles to create a new unified runfiles tree. I was more interested in just being able to pass runfiles directly into actions (my case wouldn't want to merge the runfiles). The relevant part of 15486 to this situation is the idea of being able to manually construct a FilesToRun-like object that has the triple of (executable, runfiles, runfiles_directory) (or similar), where you've specified, at the least, the executable and runfiles.