Assembly and generate script

[Paulo Coutinho]

Hi guys,

Im trying use genrule but i have some questions:

1. I have created a new package with a BUILD file and with a stub.sh that will prepend the JAR file:

Ref: https://github.com/sbt/sbt-assembly/blob/9ff6a954d8d1638573fc7b0cc0041087f17ebacb/README.md#prepending-a-launch-script

2. In generator/BUILD i put the genrule (the structure is correct?):

genrule( name = "generator", srcs = ["stub.sh", "djinni.jar"], outs = ["djinni"], cmd = "cat stub.sh djinni.jar > hello.run && chmod +x hello.run ", tools = [], )

3. When i execute bazel for the new package i get error (i know that stub.sh and djinni.jar files can be in wrong path):

bazel build //generator INFO: Analyzed target //generator:generator (0 packages loaded, 0 targets configured). INFO: Found 1 target... ERROR: /Users/paulo/Developer/workspaces/cpp/djinni/generator/BUILD:1:8: Executing genrule //generator:generator failed: missing input file '//generator:djinni.jar' Target //generator:generator failed to build Use --verbose_failures to see the command lines of failed build steps. ERROR: /Users/paulo/Developer/workspaces/cpp/djinni/generator/BUILD:1:8 Executing genrule //generator:generator failed: 1 input file(s) do not exist INFO: Elapsed time: 0.112s, Critical Path: 0.00s INFO: 1 process: 1 internal. FAILED: Build did NOT complete successfully

4. The file djinni.jar is not on generator folder, but inside "bazel-bin/src/djinni.jar", because is was generate by an other command in other package (bazel build //src:djinni). How can i reference the required files in my generator/BUILD file?

5. The genrule has an outs param that i can't omit, why it is necessary and for what i will need it? It can be a variable for my output filename here cat stub.sh djinni.jar > ${OUTS_VAR} or something like this? There is any way to reference one file from one package into the vars from other packages?

Thanks.

[Alex Humesky]

Bazel requires an "outs" parameter for genrule (and every action) so that when other actions and targets request inputs, it knows which other actions to run first. It also helps Bazel to know that actions produce what is expected.

I think the only things you need to do are specify the right inputs, and specify the right output in the command line. Something like this maybe:

genrule( name = "generator", srcs = ["stub.sh", "//src:djinni.jar"], outs = ["djinni"], cmd = "cat stub.sh djinni.jar > $@ && chmod +x $@", )

If you can build djinni with "bazel build //src:djinni", then "//src:djinni" is a label you can use in a BUILD file too. One thing to note, if //src:djinni is a java_binary target, then "//src:djinni.jar" means the jar for only the class files of that java_binary, and none of its dependencies. If you want to make a "fat" jar (one with the java_binary's classes and all the classes of its dependencies), then you'll want to use "//src:djinni_deploy.jar" instead.

"$@" is a bazel variable that means "the output file, if there is only one". (If there are multiple output files, then you have to specify $(OUTS) or each one specifically with "$(location name)".) See https://docs.bazel.build/versions/main/be/make-variables.html#predefined_genrule_variables

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/f189e6ea-1dc0-4558-b48d-88666a6f129cn%40googlegroups.com.

[ahum...@google.com]

Sorry, I missed one thing in the genrule: you'll want to use $(location name) to specify the inputs:

genrule( name = "generator", srcs = ["stub.sh", "//src:djinni.jar"], outs = ["djinni"], cmd = "cat $(location :stub.sh) $(location //src:djinni.jar) > $@ && chmod +x $@", )

So that the labels (like "//src:djinni.jar") get translated to paths. ":stub.sh" omits the package (that is, nothing before the colon), which then means "use the package this target is in".

[Paulo Coutinho]

Hi,

I solved it with this:

https://github.com/Snapchat/djinni/pull/38/files

My BUILD file:

genrule(

name = "generator",

srcs = ["stub.sh", "//src:djinni_deploy.jar"],

outs = ["djinni"],

cmd = "cat $(location stub.sh) $(location //src:djinni_deploy.jar) > $@ && chmod +x $@",

tools = [],

)

It is working. There is something wrong with $(location stub.sh) instead of $(location :stub.sh)?

Some references:

- https://github.com/bazelbuild/rules_scala/blob/b85d1225d0ddc9c376963eb0be86d9d546f25a4a/test_expect_failure/missing_direct_deps/internal_deps/BUILD#L104

- https://gitter.im/bazelbuild_rules_scala/Lobby?at=5dfbd3b1c62fdf33f71dc667

Thanks.

[Alex Humesky]

Happy to help.

$(location stub.sh) vs $(location :stub.sh)is just convention. ":file" is sometimes used to mean "generated file" where "file" means source file (i.e. file on disk). So $(location stub.sh)is fine

You received this message because you are subscribed to a topic in the Google Groups "bazel-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bazel-discuss/1Fk2J_8M4J4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/d4935096-e22e-4a4b-a104-58962616a227n%40googlegroups.com.

[Paulo Coutinho]

Hi,

One more thing:

How i can make it executable and all the params from bazel run be sent to the java executable?

genrule(
    name = "generator",
    srcs = ["stub.sh", "//src:djinni_deploy.jar"],
    outs = ["djinni"],
    cmd = "cat $(location :stub.sh) $(location //src:djinni_deploy.jar) > $@ && chmod +x $@",

    executable = True
)

When i execute bazel run the parameters are not sent to the final binary, but is recognized by bazel as bazel parameter:

bazel run //generator:generator --help
ERROR: --help :: Unrecognized option: --help

But if i call final binary generated direct, it work:

./bazel-bin/generator/djinni --help
Usage: djinni [options]

How i can pass parameters like "--help" to the binary instead of bazel when i execute `bazel run`?

Thanks.

[Alex Humesky]

For bazel run, you use "--" to separate the arguments to bazel and the arguments to the binary that bazel builds and runs, so:

bazel run //generator:generator -- --help

See https://docs.bazel.build/versions/main/user-manual.html#run

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/194f3968-bf12-4bf5-a576-f6b2cdd6ee9en%40googlegroups.com.

[Gregg Reynolds]

On Thu, Dec 23, 2021 at 1:35 PM Paulo Coutinho <pa...@prsolucoes.com> wrote:

Hi,

One more thing:

How i can make it executable and all the params from bazel run be sent to the java executable?

genrule(
    name = "generator",
    srcs = ["stub.sh", "//src:djinni_deploy.jar"],
    outs = ["djinni"],
    cmd = "cat $(location :stub.sh) $(location //src:djinni_deploy.jar) > $@ && chmod +x $@",
    executable = True
)

When i execute bazel run the parameters are not sent to the final binary, but is recognized by bazel as bazel parameter:

bazel run //generator:generator --help

[Paulo Coutinho]

Hi,

The generated jar/binary always return null on version:

Main.getClass.getPackage.getImplementationVersion

There is any way to add version to the package?

Thanks.

[Alex Humesky]

The deploy_manifest_lines attribute of java_binary can be used to add a version.

See https://docs.bazel.build/versions/main/be/java.html#java_binary.deploy_manifest_lines

java/main/Main.java

package main;

public class Main {
  public static void main(String[] args) {
    System.out.println(Main.class.getPackage().getImplementationVersion());
  }
}

BUILD

java_binary(
  name = "Main",
  srcs = ["java/main/Main.java"],
  deploy_manifest_lines = [
    "Implementation-Version: 1.2.3.4",
  ],
)

usage:

$ bazel run Main_deploy.jar
INFO: Analyzed target //:Main_deploy.jar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:Main_deploy.jar up-to-date:
  bazel-bin/Main_deploy.jar
INFO: Elapsed time: 0.124s, Critical Path: 0.00s

INFO: 1 process: 1 internal.

INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
1.2.3.4

A couple things to note:

1. The docs say "The contents of this attribute are not subject to "Make variable" substitution.", so the version has to be hard coded in a BUILD file. A way around that is to put the version in its own bzl file and then load that, and then the implementation version can be written to the bzl file on its own:

implementation_version.bzl

IMPLEMENTATION_VERSION = "1.2.3.4"

BUILD

load(":implementation_version.bzl", "IMPLEMENTATION_VERSION")

java_binary(
  name = "Main",
  srcs = ["java/main/Main.java"],
  deploy_manifest_lines = [
    "Implementation-Version: " + IMPLEMENTATION_VERSION,
  ],
)

2. The implementation version will be null if you run the binary "Main" with bazel run or use bazel-bin/Main, because that is actually a wrapper script that constructs a class path with the individual jars of the java_binary and each java_library, and the deploy_manifest_lines are written to only the deploy jar. So we have to build Main_deploy.jar and run that.

On Mon, Dec 27, 2021 at 11:35 PM Paulo Coutinho <pa...@prsolucoes.com> wrote:

Hi,

The generated jar/binary always return null on version:

Main.getClass.getPackage.getImplementationVersion

There is any way to add version to the package?

Thanks.

Em quinta-feira, 23 de dezembro de 2021 às 18:11:02 UTC-3, d...@mobileink.com escreveu:

On Thu, Dec 23, 2021 at 1:35 PM Paulo Coutinho <pa...@prsolucoes.com> wrote:

Hi,

One more thing:

How i can make it executable and all the params from bazel run be sent to the java executable?

genrule(
    name = "generator",
    srcs = ["stub.sh", "//src:djinni_deploy.jar"],
    outs = ["djinni"],
    cmd = "cat $(location :stub.sh) $(location //src:djinni_deploy.jar) > $@ && chmod +x $@",
    executable = True
)

When i execute bazel run the parameters are not sent to the final binary, but is recognized by bazel as bazel parameter:

bazel run //generator:generator --help

bazel run //generator:generator -- --help

--

You received this message because you are subscribed to a topic in the Google Groups "bazel-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bazel-discuss/1Fk2J_8M4J4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/209357a1-6abe-476b-823f-e484d7639093n%40googlegroups.com.

[Paulo Coutinho]

Hi,

First, thanks.

Second, the scala binary appear don't have support for it:

BUILD:

load("@io_bazel_rules_scala//scala:scala.bzl", "scala_binary")

load("//bzl:implementation_version.bzl", "IMPLEMENTATION_VERSION")

scala_binary(

name = "djinni",

srcs = glob(["source/**/*.scala"]),

main_class = "djinni.Main",

deps = [

"@maven_djinni//:com_github_scopt_scopt_2_11",

"@maven_djinni//:org_scala_lang_modules_scala_parser_combinators_2_11",

"@maven_djinni//:org_yaml_snakeyaml",

],

deploy_manifest_lines = [

"Implementation-Version: " + IMPLEMENTATION_VERSION,

],

visibility = ["//visibility:public"],

)

ERROR:

ERROR: /Users/paulo/Developer/workspaces/cpp/djinni/src/BUILD:4:13: //src:djinni: no such attribute 'deploy_manifest_lines' in 'scala_binary' rule
ERROR: /Users/paulo/Developer/workspaces/cpp/djinni/generator/BUILD:1:8: Target '//src:djinni_deploy.jar' contains an error and its package is in error and referenced by '//generator:generator'
ERROR: Analysis of target '//generator:generator' failed; build aborted: Analysis failed

Do you have any idea?

[Alex Humesky]

I didn't realize this was for the scala rules. I'm not too sure how the scala rules work, but it doesn't look like they support adding things to the manifest directly. I was going to suggest using the "resources" attribute and add a MANIFEST.MF file there, but it doesn't look like singlejar (the tool that's used to create deploy jars) will merge MANIFEST.MF files. You probably want to file a feature with the scala rules: https://github.com/bazelbuild/rules_scala/issues Or perhaps there's some way to do it that they can point out.

As a workaround, you could add this manually in a genrule, it's pretty hacky though:

genrule(
  name = "scala_main_with_implementation_version",
  srcs = [":scala_main_deploy.jar"],
  outs = ["scala_main_with_implementation_version_deploy.jar"],
  executable = True,
  cmd = """
cp "$<" "$@"
chmod +w "$@"
unzip "$@" META-INF/MANIFEST.MF
# singeljar appears to write an extra CR LF at the end, and java

# doesn't appear to like more entries after that,

# so remove the last CR LF
truncate -s -2 META-INF/MANIFEST.MF
echo -n "Implementation-Version: 1.2.3.4\r\n" >> META-INF/MANIFEST.MF
zip -u "$@" META-INF/MANIFEST.MF
""",
)

A lot of the hackiness is due to the "truncate -s -2", there are probably more robust ways to remove the extra CR LF, especially since java will accept CR LF, CR, or LF for newlines in the manifest.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/16f54b98-ba6a-4f67-9db3-f937ec2765c7n%40googlegroups.com.

[Paulo Coutinho]

I post the problem in the issue on Github:

https://github.com/bazelbuild/rules_scala/issues/166