I enabled incompatible_strict_action_env but my golden-test can't find "diff" binary anymore

25 views
Skip to first unread message

matt...@gmail.com

unread,
Apr 21, 2021, 5:53:59 AMApr 21
to bazel-discuss
Hi,

I've enabled `incompatible_strict_action_env` since it seems like the right thing to do if you care about reproducibility (and improve cache hits).
We use tasty-golden (https://hackage.haskell.org/package/tasty-golden) to run our golden-tests but now it fails to find the `diff` binary.

I think I could solve it by appending to PATH `--action_env=PATH=/path/to/diff:$PATH` but that seems overkill: I would rather have just the golden tests see the diff binary.

Our test is built like this (native_test comes from bazel-skylib):
===
native_test( name = test_name, src = name, out = test_name + "_", data = data, tags = tags, **test_args, )
===
and the path to diff is "@diffutils//:bin/diff". I've tried several things to no avail, the test never finds `diff`. Seems like `tools` and `deps` are only used at compilation time. I tried passing
`data = glob([ "golden-tests/in/**", "golden-tests/out/**", ]) + [ "@diffutils//:bin/diff" ] ` too with a failure as well.
Any tip ?

Best regards
Matt

NB: this is with bazel 3.7.1


Herrmann, Andreas

unread,
Apr 21, 2021, 7:56:16 AMApr 21
to matt...@gmail.com, bazel-discuss
Hi Matt,

Indeed, the --action_env flag would be one option, however, as you note it may be overkill as it will change PATH for all tests.

Correct, tools and deps are for build time dependencies. Runtime dependencies need to be declared in the data attribute. However, that alone only adds them to the runfiles tree. (https://docs.bazel.build/versions/master/skylark/rules.html#runfiles)
So, you need some way to pass the runtime path to @diffutils//:bin/diff to the test runner.
Note, generally you'll need to use the runfiles library to find that path at runtime, see here: https://github.com/tweag/rules_haskell/blob/e1628fb9db8f9a80ac23ff1ef23432b76c906096/tests/binary-with-data/bin2.hs#L10
Instead of a command-line flag you could also hard-code the path into the binary, either manually, or e.g. using CPP, similar to here: https://github.com/tweag/rules_haskell/blob/e1628fb9db8f9a80ac23ff1ef23432b76c906096/tests/binary-with-data/bin1.hs#L11
This all assumes that you have control over the test runner sources and can e.g. use goldenVsFileDiff to specify the path to the diff command.


Side note, I'm not sure I understand the need for native_test. Is haskell_test not suitable for your use-case?

Best, Andreas



--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/0d62da6e-c261-4697-81c2-1a5e39836309n%40googlegroups.com.

Matt

unread,
Apr 21, 2021, 12:39:18 PMApr 21
to Herrmann, Andreas, bazel-discuss
Mr Herrmann my guardian angel once again saves the day.
That worked perfectly thanks. I thought diff was hardcoded in
tasty-golden but I was merely looking at an example, it's indeed the
responsibility of the downstream to setup the diff binary so I just
had to patch our code with your recommendation.
Thanks a lot for going into details/sharing the code, I hope someone
can put it in a FAQ (shouldn't incompatible_strict_action_env be a
default really ?! if yes these "tricks" should be more visible).

Thanks a lot for your work on rules_haskell where time and time again
you've addresses my issues (relevant or not :) ).

Best regards
Matt


Le mer. 21 avr. 2021 à 13:56, Herrmann, Andreas
<andreas....@tweag.io> a écrit :

Matt

unread,
Apr 21, 2021, 12:40:10 PMApr 21
to Herrmann, Andreas, bazel-discuss
and I forget: no special reason to use native_test than legacy I
think, I will look into using haskell_test instead.

Herrmann, Andreas

unread,
Apr 22, 2021, 5:40:37 AMApr 22
to Matt, bazel-discuss
Thank you for the kind feedback, that is much appreciated!

Regarding docs/FAQ, that's a good point. Handling such runtime dependencies would be a good addition to the use-case documentation (https://rules-haskell.readthedocs.io/en/latest/haskell-use-cases.html) and the troubleshooting section (https://github.com/tweag/rules_haskell#troubleshooting) could be extended with a section on missing runtime dependencies in general and in case of strict-action-env that references the use-case docs. I've opened https://github.com/tweag/rules_haskell/issues/1546 to track it.

Best, Andreas

Matt

unread,
Apr 22, 2021, 5:46:24 AMApr 22
to Herrmann, Andreas, bazel-discuss
These tricks are not rules_haskell specific, they could be in the
bazel wiki/doc or a blogpost on how to enable
incompatible_strict_action_env since it seems like such a crucial
option.

Le jeu. 22 avr. 2021 à 11:40, Herrmann, Andreas
Reply all
Reply to author
Forward
0 new messages